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PREFACE 



The current intense interest in alternative forms of assessment is based on a 
number of assumptions that are as yet untested. In particular, the claim that 
authentic assessments will improve instruction and student learning is supported 
only by negative evidence from research on the effects of traditional multiple- 
choice tests. Because it has been shown that student learning is reduced by 
teaching to tests of low-level skills, it is theorized that teaching to more 
curricularly defensible tests will improve student learning (Frederiksen & Collins, 
1989; Resnick & Resnick, 1992). In our current research for the National Center 
for Research on Evaluation, Standards, and Student Testing (CRESST) we are 
examining the actual effects of introducing new forms of assessment at the 
classroom level. 

Derived from theoretical arguments about the anticipated effects of 
authentic assessments and from the framework of past empirical studies that 
examined the effects of standardized tests (Shepard, 1991), our study examines a 
number of interrelated research questions: 

1. What logistical constraints must be respected in developing alternative 
assessments for classroom purposes? What are the features of 
assessments that can feasibly be integrated with instruction? 

2. What changes occur in teachers' knowledge and beliefs about assessment 
as a result of the project? What changes occur in classroom assessment 
practices? Are these changes different in writing, reading, and 
mathematics, or by type of school? 

3. What changes occur in teachers' knowledge and beliefs about instruction 
as a result of the project? What changes occur in instructional practices? 
Are these changes different in writing, reading, and mathematics, or by 
type of school? 

4. What is the effect of new assessments on student learning? What picture 
of student learning is suggested by improvements as measured by the 
new assessments? Are gains in student achievement corroborated by 
external measures? 

5. What is the impact of new assessments on parents' understandings of the 
curriculum and their children's progress? Are new forms of assessment 
credible to parents and other "accountability audiences" such as school 
boards and accountability committees? 

This report summarizes current project findings for the parent opinion 
portion of the study. 



Frederiksen, J. R., & Collins, A. (1989). A systems approach to educational testing. Educational 
Researcher, 18(9), 27-32. 

Resnick, L. B., & Resnick, D. P. (1992). Assessing the thinking curriculum: New tools for educational 
reform. In B. R. GifTord & M. C. O'Connor (Eds.), Changing assessments: Alternative views of 
aptitude, achievement and instruction (pp. 37-75). Boston: Kluwer Academic Publishers. 

Shepard, L. A. (1991). Will national tests improve student learning? Phi Delta Kappan, 73, 232-238. 
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An Analysis of Parent Opinions and Changes in Opinions Regarding 
Standardized Tests, Teacher's Information, and Performance 

Assessments 1 

Lorrie A* Shepard 
Carribeth L. Bliem 
CRESSl/University of Colorado at Boulder 

The current debate about the virtues of standardized tests versus 
performance assessments is often confused because advocates and antagonists 
fail to specify which measurement feature they are arguing for or against: basic 
skills content, multiple-choice format, norm-referenced score interpretation, or 
standardized administration and scoring procedures. Despite the potential for 
multiple meanings we use the term "standardized test" because this is the name 
parents and policy makers use for tests like the CTBS and ITBS. For them the 
term implies external objective tests with bubble-in answers and national norms. 
We use the term performance assessment as defined by Res nick and Res nick 
(1992). "A performance assessment is one that uses direct judgments and 
evaluations of performances rather than indirect indicators of competence" (p. 
61). We consider performance assessments to be synonymous with direct 
assessment as described by Frederiksen and Collins (1989) and authentic 
assessment described by Wiggins (1989). 

A growing body of research has documented the negative effects of 
standardized achievement tests on teaching and learning. In some cases efforts 
to raise test scores have distorted curriculum to such an extent that reading and 
mathematics have been emphasized to the exclusion of science and social studies. 
More pervasively, the form of instruction has been reshaped to imitate the format 
of multiple-choice test questions. As a consequence, students have been taught to 
recognize right answers rather than to generate their own ideas and solutions to 
problems. Although standardized achievement tests may have been selected 



1 Sections of this report summarizing initial fall opinion data were presented originally by 
Shepard and Bliem (1993) at the annual .neeting of the American Educational Research 
Association. New data presented in this report include spring quesionnaire and interview 
results and fall-spring change analyses. 
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because they provided low-cost, objective scoring and national norms, it is the 
limitations of test content and multiple-choice test format that have led to these 
deleterious effects when administered in a high-stakes accountability 
environment. 

In contrast, reformers expect performance assessments to have a 
nondistorting and positive effect on instruction and learning by aiming at more 
direct, complete, and integrated measures of desired student outcomes. Then, if 
teachers attempt to improve performance by providing practice on assessment- 
like tasks, there should be no discrepancy between these activities and ideal 
instruction. Unfortunately, the potential benefits of performance assessments 
remain to date largely theoretical because large-scale implementation efforts are 
only in the developmental stages. Many technical, educational, and political 
problems are yet to be resolved including unreliability of scoring, 
nongeneralizability across tasks, corruptibility in high-stakes contexts, inequities 
in teacher preparation for new ways of teaching, etc. 

In this report, we address one political obstacle to performance 
assessment — the objection of parents. Although we do not claim, as do some 
advocates, that performance assessments are automatically valid and equitable, 
we do believe that challenging academic assessments could be powerful tools for 
improving classroom instruction. Therefore, it is worrisome if educators are unable 
to invest in development efforts for fear of a backlash from parents. 

Policy makers perceive a demand for standardized testing from both parents 
and the general public. Standardized test score results are believed to be essential 
to monitor individual student progress and to hold schools accountable. For 
example, this demand is evidenced by Gallup Poll results demonstrating the high 
percentage of respondents favoring the use of standardized national tests by 
public schools in their community to measure the academic achievement of 
students (Elam, Rose, & Gallup, 1992). Because of this perception, legislators, 
school board members, and professional educators fear that parents will be "up in 
arms" if performance assessments are adopted to supplant familiar standardized 
tests. In the early stages of the current research project, for example, 
sympathetic members of the district accountability committee asked that 
researchers discontinue the use of the phrase "alternative assessment" because it 
connoted abandonment of standards and rigor. We adopted the term performance 
assessment in deference to their wishes. 



The purpose of this study was to examine more systematically parent 
opinions about standardized tests and performance assessments. We were 
especially interested in focusing parent attention on the content and form of these 
two types of measures, but we also provided parents with the opportunity to talk 
about the standardization and norm-referenced features of traditional tests. The 
research was guided by the following questions. 

x. How do parents in this sample respond to marker Gallup Poll questions 
about the quality of local schools, the quality of information provided to parents, 
and the desirability of standardized national tests? 

2. How do parents evaluate the usefulness of standardized tests compared to 
less formal types of information such as report cards, talking to the teacher, or 
seeing samples of their child's work? Do parents differentiate between information 
they find useful in learning about their child's progress in school versus the 
information they use to make judgments about the school as a whole? 

3. After being shown sample questions and tasks, what preferences do 
parents express regarding district use of standardized tests and performance 
assessments in reading and mathematics? Is it true that parents disdain the use 
of performance assessments as less rigorous or objective? What features of the 
two types of measures affect parent preferences? 

4. How do parents respond when asked to make their judgments about 
standardized-test questions and performance assessments in the context of 
classroom instruction? 

Methods 

Parent Sample 

Questionnaire parent samples were selected in both the participating and 
control schools. The sampling design was developed such that mutually exclusive 
but randomly equivalent stratified samples of parents could be used to measure 
change in parent opinions over time. One-third of the parents were sent 
questionnaires in the fall and a new one-third were sent questionnaires at the end 
of the school year. (In the participating schools only, the interview samples were 
drawn from the remaining one-third of parents.) In the fall, questionnaire 
responso rates were 70% for the participating schools and 49% for the control 



schools resulting in samples of 69 and 36 respectively. In the spring, response 
rates were 31% and 45% for participating and control schools. 

For the first round of interviews, parents of three children per classroom in 
the participating schools were sampled to be interviewed in conjunction with 
first-quarter parent conferences. Children were stratified by teacher-judged 
achievement level (high, middle, and low). Teachers were provided with two 
alternate names within each stratum to use in cases where parents declined to 
participate or where scheduling precluded particip; ttion. We have no accurate 
way to estimate what biases were introduced into the sample by refusals or 
schedule conflicts. However, it was a relatively raie occurrence for teachers to 
pick the second name on the list because a conference was going to be problematic 
or because a parent was thought to be unresponsive to .school requests. Ironically 
a greater percentage of parents were willing to commit tc the time demands of the 
interview than returned questionnaires. After the fall interview sample was 
agreed upon (n = 13 classrooms x3 = 39), attrition occurred in the sample because 
of absence from the conferences (n = 5) and technical failures in recording (n = 1), 
resulting in a final interview sample of 33 parents or parent dyads. Absence from 
the scheduled parent conferences was related to student achievement level; 3 of 
the absent parents had children in the low ability group and 2 were from the 
middle group. Therefore, the final sample was* not as balanced as originally 
intended. 

Ideally the second round of parent interviews would have been conducted at 
the end of the school year after parents had had sufficient time to become familiar 
with the assessments being used in their child's classroom. However, the 
opportunity to interview in conjunction with parent conferences was constrained 
by each school's unique schedule. The second round of parent interviews was 
conducted at the end of the second quarter in one school, at the end of the third 
quarter in another, and at the end of the second trimester in another. Attendance 
even at agreed-upon conferences was not so good at mid-year as in the fall, 
possibly because of reduced interest or because of winter snow storms. From an 
initial sample of 39 only 27 interviews were conducted. 

Questionnaire and Interview Instruments 

The four-page mailed questionnaire (Appendix PI) had three parts. The first 
part asked parents to rate the usefulness of report cards, talking to the teacher, 



standardized tests, and seeing graded samples of their child's work. The second 
part provided displays showing typical questions on third-grade standardized 
achievement tests and typical performance assessment tasks in both reading and 
math. Parents were asked to rate their degree of approval for each of the four 
types of measures. Then they were asked to show their preference for one or the 
other type of measure when used in instruction. The third set of questions was 
taken from the 1992 Gallup Poll. Parents responded to the questionnaires 
anonymously. 

Interviews, conducted immediately following parent conferences, were guided 
by a structured but open protocol (Appendix P2). The interview was divided into 
three parts, two of which closely paralleled the parent questionnaires. In the first 
and unique part of the interview, parents were asked broad questions aimed at 
eliciting their conceptions of reading and mathematics knowledge and what they 
used as indicators to judge their child's progress in each area. Two forms of the 
interview were printed so that half of the interviews began with reading questions 
and half began with mathematics. The second set of questions asked parents to 
rate the usefulness of report cards, talking to the teacher, standardized tests, and 
seeing graded samples of work. Parents were asked to think aloud and elaborate 
on their answers while filling out the rating form. 

For the last set of interview questions, parents were shown more extended 
examples of questions from standardized tests and performance assessments in 
reading and mathematics. The actual sample pages used to exemplify 
standardized test questions are shown in Figures 1 and 2. The sample 
performance assessment questions used for mathematics are shown in Figure 3. 
The sample performance assessment used in reading is not shown. It was a 15 
page booklet with a complete story, attractive line drawings, and several 
open-ended questions with multiple formats, including drawing a picture. Parents 
were asked about their approval or disapproval of each type of measure and about 
the advantages or disadvantages they saw with each. A final question asked 
parents which of the two kinds of tests they would like to see used in classroom 
instruction. 

Interviews were audio recorded and transcribed. Extensive field notes were 
also taken. In two cases where parents declined to be recorded, answers were 
typed directly into a computer. 



Examples of Questions 



Figure 1 

on Standardized Achievement Tests in Reading 



Vocabulary 



Comprehension 



1 . under the table 

O below 

O next to 

O away from 

O on top of 

2. pleasant smile 

O silly 

O wicked 

O cheerful 

O plain 

3. frighten away 

O free 

O scare 

O run 

O finish 

4. talked quietly 

O funny 

O a lot 

O quickly 

O softly 



Danny began his strange occupation by 
accident. He was trying to think of a way to make 
money when his friend Eddie came over. Eddie said 
his family couldn't leave for their trip until they found 
someone to take care of their pet snake. The people 
who took care of their dog hated snakes. Eddie 
wondered if Danny would take care of the snake. 
Danny was excited but he was sure that his mother 
wouldn't let him take the job. But she surprised him 
by saying that he could. 

After that first job, Danny put a sign in the pet 
shop. It said he would take care of snakes for a fair 
price. Because snakesitters were hard to find and 
usually cost a lot, Danny got many jobs. Danny had 
found an unusual way to make money. 

(from Silver Burdett & Ginn's 
Garden Gates workbook, 1990) 

5. A good title for this story would be - 

O The Pet Store 

O A Strange Occupation 

O Going on a Trip 

O Danny's Pet Snake 

6. What did Eddie want Danny to do? 

O take care of his dog 

O work in a pet shop 

O go on a trip 

O take care of his snake 



7. Why did Danny get many jobs? 

O because snakesitters cost very little 

O because snakes like to go on trips 

O because snakesitters are hard to find 

O because Eddie went on many trips 



Figure 2 

Examples of Questions on Third-Grade Standardized Achievement Tests in Mathematics 



Computation 



Concepts and Applications 



Add. 
1. 



+ 



3842 
4104 



Subtract. 
2. 



82 - 3 = 



Multiply. 
3. 



6 

x 9 



Divide. 
4. 



8 mr 



O 7946 

O 7746 

O 7906 

O 7942 

O None of these 



What is 763 rounded to the nearest 
hundred 



o 
o 
o 
o 



700 
750 
760 
800 



O 89 

O 79 

O 81 

O 52 

O None of these 



How much change will you get if 
you have $6.55 and spend $4.32? 



o 
o 
o 
o 



$2.23 
$2.43 
$3.23 
$10.87 



What is the perimeter of this shape? 
6 ft 



O 63 

O 48 

O 54 

O 69 

O None of these 



3 ft 



O 2 

O 24 

O 20 
O 8 

O None of these 



4 ft 



2 ft 



O 20 ft 

O 21 ft 

O 22 ft 

O 23 ft 



5 ft 



2 ft 
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Figure 3 

Examples of Third-Grade Performance Assessment Questions 



1. Bus Ride - A friend of yours, who just moved to the United States, must ride the bus to 
and from school each day. The bus ride costs 50 cents. Your friend must have exact 
change and must use only nickels, dinrvas, and quarters. Your friend has a problem because 
she does not yet understand our money, and she does not know how to count our money. 

Help your friend find the right coins to give to the bus driver. Draw and write 
something on a whole sheet of paper that can help her. She needs a sheet of paper that 
can show which combinations ot coins can be used to pay for the 50-cent bus ride. 



Sample Student Answer 1 



Sample Student Answer 2 





JO 




o 


0 






"V. 


I 




/ 


1 


m 


O 


1 


O 


5" 


o 




if 


























rM 



JO =-<Ji^vC- - 

do in 

I * *uK (a. 





1221 



i 



1 



50* 



2. 



For the figure at left, show 1/2 in as many ways 
as you can. You may draw more figures, if 
necessary. For each way you find, explain how 
you know you have 1/2. 



3. Suppose you couldn't remember what 8 x 7 is. How could you figure it out? 



4. Our class of 26 students is going to the Denver Art Museum. How many cars do we 
need if 4 students can go in each car? How many do we need if only 3 students can go in 
each car? 



Note: The first two examples aro reproduced with permission from Pandoy. T. (19911. A Sampler ot Math ematics 
Assessment , Sacramento. CA: Calilornia Department of Education. 
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Figure 3 (continued) 



5. Which spinner is most likely to spin a sum greater than seven after two spins? Put a circle 
around your choice. 







Explain your choice. 



Which one of the bar graphs below shows the 
same data as the circle graph on the right? 
Put a circle around your choice. 







Explain your choice. 



Adam says that 4 + 52 is 452. Is Adam right or wrong? What would you tell Adam? 



,8. Put 4 different one-digit numbers in the boxes to make the largest possible answer. 



+ □ 



How did you know what to choose? 
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Findings 

The descriptive analysis is based primarily on data collected in the fall 
because these data were largely unaffected by the assessment development 
project and therefore are more likely to be generalizable to the opinions of parents 
in other school districts. Then in a latter section, evidence of change from fall to 
spring, or more importantly differential change in participating schools, is 
considered. 

Gallup Poll Results 

Nationally 71% of the general public as well as 71% of public school parents 
favor requiring public schools in their community to use standardized national 
tests (Elam et al., 1992). If the purpose of the present study is to go beyond this 
superficial endorsement of standardized tests and develop a deeper understanding 
of parent opinions, then it is important to know how similar parents in the study 
sample are to the national population of parents. 

Table 1 provides a summary of questionnaire results for third-grade parents 
in the study compared to data from the 1992 Gallup Poll (Elam et al., 1992). A 
preliminary analysis showed no significant differences between questionnaire 
responses for control and participating schools, therefore they are combined here 
as a single sample. 

On the most classic Gallup question (#21) concerning national tests, there is 
a disconcertingly large disparity between third-grade parents in this study and the 
national sample of public school parents. Only 46% of local third-grade parents 
favor national tests compared to 71% nationally. Opposition rates are more 
similar because a very large proportion of parents (25%) in the study sample 
responded in the don't-know category. On other questions, the gap between the 
two groups is much narrower suggesting that the third-grade sample can be 
thought of as only slightly more negatively disposed toward standardized testing 
than the nation as a whole. On questions 22-27 addressing various uses of 
standardized national tests, the percentage of third-grade parents endorsing each 
use was consistently 12% to 14% less than for the national group, with the 
exception of using test results to improve teaching where the difference was only 
7%. 

15 
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Table 1 

A Comparison of National Results With Fail Parent Questionnaire Responses (n=105) on 1992 Gallup 
Pol! Questions 

18. Students are often given the grades A, B, C, D, and Fail to denote the quality of their work. Suppose the 
GiMc schools themselves, in this community, were graded in the same way. What grade would you give 
the public schools here— A, B, C, D, or FAIL? 







A 


B 


C D FAIL Don't Know 


• 


1992 National % 


9 


31 


33 12 5 10 




3rd Grade Parent Sample % 


13 


53 


25 2 2 5 




19. How about the public schools in the nation as a whole? 


What grade would you give the public schools 




nationally— A, B, C, D, or FAIL? 










A 


B 


C D FAIL Don't Know 


• 


1992 National % 


2 


16 


48 18 4 12 




Public School Parents National % 


2 


17 


48 18 4 11 




3rd Grade Parent Sample % 


1 


8 


50 20 7 15 



20. What grade would you give the information you receive from your third grade child's teacher(s) regarding 
his or her academic progress — A, B, C, D, or FAIL? 



• 




A 


B 


C 


D 


FAIL 


Don't Know 




1992 National % 


31 


30 


23 


7 


6 


3 




Public School Parents National % 


42 


29 


20 


6 


2 


1 




3rd Grade Parent Sample % 


32 


37 


20 


6 


1 


4 



21 . Would you favor or oppose requiring the public schools in this community to use standardized national 
tests to measure the academic achievement of students? 

1992 Public School Parents 3rd Grade 
National % National % Parent Sample 

Favor 71 71 46 

Oppose 20 23 30 

Don't Know 9 6 25 

In addition to measuring the academic achievement of students, do you think these standardized national tests 
should be used or should not be used for the following purposes? 

1992 National % 3rd Grade Parent Sample 







Should 


Should 
Not 


Don't 
Know 


Should 


Should 
Not 


Don't 
Know 


22. 


To rank the local public schools in 
terms of student achievement 


65 


26 


9 


53 


37 


10 


23. 


To determine if a student advances to 
the next grade level of schooling 


60 


32 


8 


46 


45 


9 


24. 


To determine how much teachers 
should be paid 


38 


52 


10 


26 


62 


12 


25. 


To determine the level of funding each 
local school should receive 


36 


54 


10 


22 


68 


10 


26. 


To identify areas in which teachers 
need to improve thoir teaching skills 


79 


14 


7 


72 


17 


11 


27. 


To identify areas in which students 
need extra help 


85 


9 


6 


73 


19 


8 



11 
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On the more general Gallup questions dealing with quality of schools and 
quality of information provided by schools about each child's academic progress, 
the third-grade parents rated their schools higher (66% gave As and Bs compared 
to 40% for public school parents nationally); they rated the nation's schools lower 
(9% As and Bs vs. 19% nationally); and they rated the quality of information they 
receive just the same as did a national sample of elementary parents (69% vs. 
71% As and Bs). 

Given the above similarities *nd dissimilarities between national samples of 
parents and the third-grade sample here, we suggest that the findings in this 
study — about broad parent preferences and especially about reasons parents give 
for their preferences — are likely to be generalizable beyond the confines of this 
particular district. However, it is unlikely that strict quantifications, such as the 
percent preferring performance assessments over standardized tests, can be 
generalized either to nonrespondents in the same district or to the nation as a 
whole. To serve as a check on the local district's relative bias against standardized 
tests, we replicate some of the key analyses below using only the subsample of 
parents who favored standardized national tests on the Gallup Poll question. 

Parent Opinions About Standardized Tests and Informal Sources of 
Information: Questionnaire Results 

On the questionnaire, parents were asked to evaluate the usefulness of four 
types of information for two different purposes, learning about their child's 
academic progress and judging the quality of the school. Data in Table 2 show the 
percentage responses for the first purpose. Overwhelmingly, parents believe that 
they learn the most about their child's progress by listening to the teacher; 77% 
rated this source of information as very useful. A total of 94% rated hearing from 
the teacher in the top two categories. All three informal sources of information 
about student progress — report cards, talking to the teacher, and work samples — 
received strikingly higher usefulness ratings than did standardized tests. 
However, it should be noted that a biasing factor in this study is that third-grade 
parents were surveyed in the fall before the first opportunity for their third-grade 
child to be tested. Therefore, parents were likely to be responding on the basis of 
experience with older children or a general opinion they held regarding standardized 
tests. Parents who had little knowledge of standardized tests and whose oldest 
child was in third grade may not have had sufficient information to judge the 
usefulness of such tests. 



ERLC 



12 



17 



Table 2 

FALL Parent Questionnaire Ratings of the Usefulness of Each Type of Information for Learning 
About Their Child's Progress in School (n=105) 



Place a check in the appropriate 
column to indicate the usefulness 
of each type of information. 


Not at 

all 
useful 








Very 
useful 


Blank/ 
Missing 


1 


2 


3 


4 


5 




1. Report cards 


2% 


2% 


20% 


33% 


43% 




2. My child's teacher talking about 
his/her progress 


0 


2% 


4% 


17% 


77% 




3. Standardized tests 


6% 


15% 


41% 


22% 


14% 


2% 


4. Seeing graded samples of my 
child's work 


0 


0 


10% 


30% 


60% 





5. Pick one type of information you think is most helpful in learning about your child and tell what you learn 
'from it and why it is helpful. 

Report cards (n=5) 

"I think the most important item is report cards because it shows how well they are doing in each subject 
ard then we can determine where help is needed." 

Teachir talking (n=44) 

"* alking with my child's teacher is most helpful because I learn firsthand what progress is being made in 
c/ass, where the shortcomings are and how I can best help at home." 

Standardized tests (n=0) 

Graded samples (n=38) 

This way I can see the actual work, the teacher's response, and evaluate what I understand the child's 
level of learning to be at." 

"When we are able to see our child's work we can see the progress that he makes and can help him in the 
areas that he needs help." 

Other (n=9) 



In Table 3 parent ratings are presented for the same four types of 
information but for the purpose of judging the quality of the school. As expected, 
more parents considered standardized tests to be useful for this purpose than for 
learning about the progress of individual children (45% vs. 36%). In addition, 19 
respondents (18%) wrote in comments suggesting that it is the comparative 
information provided by such tests that makes them useful for evaluating the 
school's performance. 



Table 3 



FALL Parent Questionnaire Ratings of the Usefulness of Each Type of Information to Evaluate the 
Quality of Education Provided at Their Child's School (n=1 05) 



Place a check in the appropriate 
column to indicate the usefulness of 
each type of information. 


Not at 

all 
useful 








Very 
useful 


Blank/ 
Missing 


1 


2 


3 


4 


5 




6. Report cards 


18% 


11% 


35% 


17% 


16% 


3% 


7. My child's teacher talking about 
his/her progress 


4% 


2% 


18% 


28% 


46% 


3% 


8. Standardized tests 


8% 


14% 


30% 


20% 


25% 


4% 


9. Seeing graded samples of my 
child's work 


1% 


4% 


20% 


27% 


46% 


3% 



10. Pick one type of information you think is most helpful in learning about the quality of education provided 
at your child's school and tell what you leam from it and why it is helpful. 

Report cards (n=0) 
Teacher talking (n=26) 

Talking with the teach allows you into the school environment to measure emphasis, attitudes, some 
insight into treatment of students (i.e., respect, encouragement, etc.)." 

"Again I would say talking with the teacher in person. That way I get an understanding of the school 
policies and programs and what curriculum the school uses." 

Standardized tests (n=19) 

"Standardized tests show me where [my school] ranks in comparison with district, state and national 
averages — " 

Graded samples (n=18) 

"Again, samples of my child's work show me what it is they are doing in the classroom, and will show me 
where the school's emphasis is at." 

Other (n=25) 

"Newsletters from school sent home with the student. These discuss what the goals are that have been 
established, what type of work the students are doing, what teachers expect, etc." 

"Being able to be in the classroom, getting to know the teacher, seeing hands-on, spending time and 
helping at school." 



However, much of the data in Table 3 are contrary to the expectation that 
parents would prefer external and formal measures for school accountability 
purposes. Indeed the greatest number of parents, 84% and 83% respectively, 
valued "my child's teacher talking about his/her progress" and "seeing graded 
samples of my child's work" to help them evaluate how good a job the school was 
doing. Furthermore, in follow-up comments parents gave justifications that 



showed they were mindful of the change in purpose from the first set of questions 
to the second. Parents said that these informal sources help them learn about the 
quality of education by giving them first-hand information about the school 
curriculum, teacher knowledge and expectations, school policies, and classroom 
climate. 

Given earlier differences found between the study sample and the national 
population on Gallup questions, we were concerned that the above, somewhat 
surprising findings — downplaying the usefulness of standardized tests — might be 
true only for parents in this sample and not generalize to a broader population of 
parents. Therefore the present sample was subdivided on the basis of Gallup Poll 
question 21 into those parents who "Favor" use of standardized national tests and 
those who "Oppose" (parents answering Don't Know were omitted). Predictably 
these two groups differed in their ratings of the usefulness of standardized tests on 
question 3 in Table 2. However, they did not differ at all in their much higher 
ratings of informal sources of information. For example, on question 2, teacher 
talking, the mean ratings were 4.65 and 4.74 on the 5-point scale for the Favor 
and Oppose groups respectively. For question 4, seeing graded samples of work, 
the means were 4.58 and 4.45. (Neither difference is statistically significant and 
the slight numeric differences are offsetting.) On the second set of questions 
dealing with accountability purposes, the two groups again differed significantly in 
their usefulness ratings for standardized tests (Favor X = 3.94; Oppose X = 2.80). 
They did not differ in their ratings on the other three sources of information; both 
groups gave report cards low marks for this purpose but rated teacher talk (Favor 
X = 4.02; Oppose X = 4.10) and work samples (Favor X = 4.26; Oppose X = 
4.00) highly. This means that even the group favorable to standardized testing, 
which is represented in greater proportion in the national population, only 
considered standardized tests to be as useful as talking to the teacher for 
accountability purposes and not as useful as seeing graded work samples. For the 
purpose of learning about an individual child's progress both groups agree strongly 
that all three sources of information are more useful than standardized tests. 

Parent Opinions About Standardized Tests and Informal Sources of 
Information: Interview Results 

In the interview data, parents reported even more glowing opinions about the 
usefulness of informal types of information for learning about their child's 



academic progress than they had on the questionnaire. In Table 4a and Table 5, 
quantitative results are presented from the interview questions corresponding to 
the questionnaire data shown in Table 2. The question about standardized tests 
was pulled out and treated in a separate table to allow for separate analysis of the 
qualitative data. Although the parent responses about the usefulness of 
standardized tests are virtually the same (36% of the questionnaire sample said 
they were useful vs. 33% in the interview sample), the interview respondents were 
even more positive about the usefulness of report cards, hearing the child's 
teacher talk about progress, and graded samples of work than the questionnaire 
respondents had been. For example, 77% had rated teacher talk as "very useful" 
on the questionnaire, whereas 91% of the parents interviewed gave the "very 
useful" rating. Because parents in the interview setting were asked to fill out a 
rating sheet identical to the question set used on the questionnaire, it is unlikely 
that the demand characteristics of the interview (having to explain their answers, 
etc.) account for the more extreme answers in the interview. The more likely 
explanation is that parents were interviewed immediately following parent 
conferences. It is also possible that the interview and questionnaire data differ 
because they are based on different samples; the interview data were collected 
only in participating schools as compared to the questionnaire data collected from 
both control and participating school parents. However, there were no significant 
differences in the questionnaire results between the two types of schools on these 

Table 4a 

FALL Parent Interview Ratings Regarding the Usefulness of Informal Types of Information for 
Learning About Their Child's Progress in School (n=33) 



How useful is each type of information listed below in helping you learn about your child's 
pro gress In schooi ? While you're doing that, please tell why you think each type of 
information is useful or not very useful. 





Not 
very 
useful 








Very 
useful 


Blank/ 
Missing 


1 


2 


3 


4 


5 




a. report cards 


n=0 


1 

(3%) 


6 

(18%) 


5 

(15%) 


20 
(61%) 


1 

(3%) 


b. talking with the teacher 


0 


0 


0 


2 

(6%) 


30 
(91%) 


1 

(3%) 


d. seeing graded samples of my 
child's work 


0 


0 


0 


5 

(15%) 


27 
(82%) 


1 

(3%) 
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Table 4b 



FALL Parent Interview Responses Regarding the Usefulness of Informal Types of Information for 
Learning About Their Child's Progress in School (n=33) 

Note. Data from the three questions in Table 4a were treated as an aggregate. The following themes 
emerged from the parent comments. Categories below are not mutually exclusive. 



Nearly three-quarters of the respondents (73%, n=24) appreciate how these types of 
information provide feedback about how well their child is doing academically. They cite 
understanding their child's strengths and weaknesses as well as seeing evidence of his or 
her progress over time. 

"If i didn't have the report card, there would be no way I'd be able to judge that he was actually being able to 
retain what he's learning. She [the teacher] shows me what I really need to work on with him and with him 
being there, he sees that he needs to work on this." 

The report cards are very useful. You can [see by the] quarterly report the improvement she's made or if 
she's not making improvement, you can teli." 

"As long as she [the teacher] can see in math that he's improving with each of his tests, that's all she cares 
about. It's just as important that he is doing the best he can do, and that he's improving." 

"I like [graded samples]. I think it's useful because then I know from [the teacher] showing from September 
to now, I know that she's improving." 

Over one-third of the respondents (36%, n=12) comment on the normative aspects of this 
information. They depend on the teacher to provide the comparative information. 

"I think report cards are very useful because their pluses and minuses [system of grading] tell you where 
they're at in reference to the rest of the class basically." 

"[Graded samples] give you more of a clear idea of where the child stands within the class itself." 

"Well, [these types of information] let me know that they are doing their level of work. I know that they're at 
the level that they should be doing. That assures me that they are doing the level that they're supposed to 
be." 

"Well, I think on the report cards,, it's very useful because in her reading you know what level she is reading." 

With regard to talking to the teacher, respondents value the opportunity for 
communication with the teacher (18%, n=6). 

"If there's questions, just keeping communication with the teacher is important. If you have questions, 
sometimes things come home that you don't understand and you need to know the answers." 

Talking with the teacher, I think, is most important. We can then ask questions as to the report card..." 

Talking with the teacher helps 24% of the parents (n=8) because the teacher spends time 
with their child. 

They [teachers] see what work they're doing more than we do because we both work. They're with our kids 
9 hours a day." 

"She's the trained professional, she knows what to look for if something should come up that we should be 
aware of ...To be able to talk with somebody who can see their development and be there ail times is very 
important." 

"She deals with my child every day. She knows my daughter and can tell me things about my daughter's 
learning that she notices. She sees her in a context where she's maybe one-on-one at times, she's in a 
small group, she's in a large group, and sees my daughter in a wide variety of experiences, where I see her 
only in a family setting." 

They can see everything that they're doing, and they can see if they're messing around, not getting their 
assignments, or if they're struggling to understand..." 
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Table 5 



FALL Parent Interview Responses Regarding the Usefulness of Standardized Tests for Learning 
About Their Child's Progress in School (n=33) 



How useful is each type of information listed below in helping you learn about your child's 
progress in school ? While you're doing that, please tell why you think each type of 
information is useful or not very useful. 





Not at all 
useful 








Very 
useful 


Blank/ 
Missing 


1 


2 


3 


4 


5 




c. standardized tests 


n=4 
(12%) 


7 

(21%) 


9 

(27%) 


7 

(21%) 


4 

(12%) 


2 
(6%) 



Normative Information n=4 (X = 4.7) 

"It compares him to everybody else and lets you know how he's doing compared to them. Because if he's 
just getting As on everything and everybody else is getting As for not doing anything at all, then it doesn't 
say much for him." 

u How they do compared to other children is very important so that you have an idea that they are 
progressing like they should." 

Helpful Academically n=4(X = 4.2) 

"I think they are very useful. Not to me but to the teachers. I think they probably should have these so the 
teachers can inform the parents as to where they're at. Informs me as to what she needs help on." 

Other Positive Attributes n=3 (X = 4.0) 

Children as Individuals n=6 (X =3.0) 

"Some children just do not respond weli to certain standardized tests. I think that children are different 
types of learners and standardized testing sometimes does not always catch that, the full breadth of a 
child's learning abilities." 

"A standardized test really doesn't center around what the problems are with the individual child, but are 
used as a measurement of how much this child knows." 

Invalid Results n=6 (X = 2.7) 

1 think it's got a lot to do with how well a child takes tests, what their particular morning was like, or whether 
they had a good night sleep. I think there's so many variables that can influence their score on the 
standardized testing." 

Parents' Own Experiences n=2(X = 1.5) 

U I don't like them. And I probably have to say because when I was a kid th9y made me nervous. I do better 
without the test. And when they go to give you that big test, it's overwhelming and you know you're being 
graded on whether you're stupid or smart." 

Don't Know n=2 (X =2.5) 

Child hasn't taken test or Parents have never seen results n=3 (X =2.0) 
Other negative attributes n=2(X = 1.0) 

Unfamiliar with standardized tests n=1 (parents left question blank) 



questions. Another possibility is that willingness to participate created different 
samples for the questionnaire and interview studies; but ironically, the more 
extreme, positive responses came from the more inclusive sample. 




Transcript analyses of parent comments explaining their ratings are 
summarized in Table 4b and Table 5. In Table 5, parent comments about why 
standardized tests are useful or not useful are related predictably to their ratings 
(means are shown in parentheses). Parents who rated standardized tests highly 
commented on the usefulness of normative comparisons and the ability of tests to 
identify academic strengths and weaknesses. Parents who gave lower ratings to 
standardized tests tended to comment on the vagaries of test performance on any 
given day or their own negative experiences taking standardized tests. 

Parents had a great deal to say about why they valued informal sources of 
information, especially talking to the teacher, to learn about their child's progress. 
Note that all three informal types of information, even report cards, which got the 
lowest rating, were rated as more useful than standardized tests. Parents want to 
know in some detail how their child is doing. They want to know strengths and 
weaknesses and what they can work on at home. To interpret the meaning of 
where their child is, they seek comparative information, that is, how their child is 
doing in relation to grade-level expectations or compared to the rest of the class. 

Perhaps the most interesting finding, however, is that parents, by and large, 
trust their child's teacher to provide them with this type of information. "She's the 
trained professional, she knows what to look for . . ." "She deals with my child 
every day. She knows my daughter and can tell me things about her learning." 
We considered the very high approval ratings given to teacher information to be 
striking because usually the public endorsement for standardized national tests is 
taken to mean that parents require external verification of progress and cannot 
rely on teacher judgment. 

When the purpose shifts from learning about my child's progress to 
evaluating the school, the interview results parallel the questionnaire findings but 
again with more extremely positive ratings for informal sources of information 
from the interviewed parents. Quantitative ratings are shown in Tables 6a and 7. 
Just as with the questionnaire data, the usefulness of standardized tests 
increased when the purpose was to judge the school rather than an individual 
child's progress. On the questionnaire the increase in positive ratings was from 
36% to 45%; for the interview sample the increase was from 33% to 45%. Again on 
the question about standardized tests, there was a close correspondence between 
the responses from the questionnaire and interview samples. In explaining their 
ratings, approximately one-third of the parents interviewed said something about 
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Table 6a 

FALL Parent Interview Ratings Regarding the Usefulness of Informal Types of Information for Learning 
About the Quality of Education Provided at the School (n=33) 

How useful is each type of information listed below in helping you learn about the quality 
of education provided at this school ? While you're doing that, please tell why you think 
each type of information is useful or not very useful. 





Not very 
useful 








Very 
useful 


Blank/ 
Missing 


1 


2 


3 


4 


5 




a. report cards 


n=4 

(12%) 


6 

(18%) 


9 

(27%) 


4 

(12%) 


8 

(24%) 


2 
(6%) 


b. talking with the teacher 


2 
(6%) 


1 

(3%) 


4 

(12%) 


2 
(6%) 


23 
(70%) 


1 

(3%) 


d. seeing graded samples of 
my child's work 


2 
(6%) 


1 

(3%) 


3 
(9%) 


8 

(24%) 


18 

(55%) 


1 

(3%) 



the value of standardized tests for making comparisons. "That is one of the 
reasons I like the standardized tests, because to me if you have a national 
standard test for third graders, it shows you where your kid is against national 
standards which doesn't necessarily say anything about your kid but it might 
point out there is a problem here . . ." 

When rating informal sources of information the results from the interviews 
can be seen as a more extreme version of the pattern found in the questionnaire 
data (as occurred for the first set of questions on children's progress). Even for the 
purpose of evaluating the school, parents find talking to the teacher and seeing 
graded samples of work to be more useful than standardized tests. In Table 6b 
categories of parent responses are presented with illustrative quotations. In 
support of their positive ratings, parents explained that talking with the teacher 
allowed them to see what expectations were being set, the quality of the 
curriculum, and how caring the teacher is with students. Specifically parents said 
that seeing the actual work that students brought home let them judge whether 
what was being taught was worthwhile. 

"I can see what kind of work the teacher is handing out. The teacher is the 
one that's in there quarter-backing the classroom. You know, if she's handing out 
pretty basic stuff to the kids to work on, then that's pretty boring, you know, 
'get-me-through-the-school-day' tyne of activities as far as I'm concerned. But if 
she's handing out stuff that will keep their interest and get their initiative going as 
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Table 6b 



FALL Parent interview Responses Regarding the Usefulness of Informal Types of Information for 
Learning About the Quality of Education Provided by the School (n=33) 

Note. Data from the three questions in Table 6a were treated as an aggregate. The following themes 
emerged from the parent comments. Categories below are not mutually exclusive. 



Talking with the teacher helps identity her expectations and goals (n=4) 

Talking with them gives you a better feel of the goals that they have in mind for the kids, the criteria, 
teacher criteria, the goals for different grade levels. Usually when we talk about their progress it's In 
relationship to those kind of things which is an indication to me of the quality of the education here." 

Talking with the teacher is telling about her personality and style with students (n=7) 

"I think if there's a lot of kids that are having problems, that maybe it reflects the teacher. You know, so to 
me, that's where I would say how it affects the school." 

"I think that's very useful, very significant to tell you that the teachers are really caring, or that they're 
really good with the students." 

Talking with the teacher tells me a great deal about the quality of education because she is able to tell me 
about herself, you know, her strengths, her weaknesses, and things like that." 

Seeing graded samples (and to a lesser extent, talking with teacher) allows an evaluation 
of the curriculum (n=15) 

That [graded samples] lets me know whether my child's learning and I think the paperwork that they get 
lets me know what they're doing." 

tt I think [graded samples are] very useful. I like to know what kind of things because not everything that 
they teach, you think is useful." 

M l just think if it's a really good cross section of what they do, the samples, then it would be a pretty good 
indication of the quality. If it's just random cases of their work, or ones that they just save becau* e they 
like and they put them in the portfolio, or something, then I don't think it's a real good indication." 

"I can see what kind of work the teacher is handing out. The teacher is the one that's in there 
quarterbacking the classroom. You know, if she's handing out pretty basic stuff to the kids to work on, 
then that's pretty boring, you know, 'get-me-through-the-school-day' type of activities as far as I'm 
concerned. But if she's handing out stuff that will keep their interest and get their initiative going as far as 
keeping them active in school, and wanting to learn, that pretty much sets the tone for the school year and 
gives me an indication of what kind of quality teachers there are, and what kind of quality programs are 
here at this school." 

Report Cards are not useful because they're about the child (n=3) 

That's a 2 because that doesn't tell anything about the school, that tells about the student." 

Report Cards are not useful because of subjective grading (n=4) 

"[They] wouldn't be very useful in telling the quality of education because you're not really sure what they 
are being graded on until it has been explained to you or you have see other things." 

"...because everybody grades differently." 
Report Cards are not useful— Other reasons (n=10) 



far as keeping them active in school, and wanting to learn, that pretty much sets 
the tone for the school year and gives me an indication of what kind of quality 
teachers there are, and what kind of quality programs are here at this school." 
The only consistently negative comments regarding informal measures pertained 
to report cards. About half of the sample said that report cards were not very 



Table 7 



FALL Parent Interview Responses Regarding the Usefulness of Standardized Tests for Learning 
About the Quality of Education Provided by the School (n=33) 

How useful is each type of information listed below in helping you learn about the quality 
of education provided at this school ? While you're doing that, please tell why you think 
each type of information is useful or not very useful. 





Not at all 
useful 








Very 
useful 


Blank/ 
Missing 




1 


2 


3 


4 


5 




c. standardized tests 


6 

(18%) 


2 
(6%) 


8 

(24%) 


7 

(21%) 


8 

(24%) 


2 
(6%) 



Normative information n=12 (X=4.2) 

"The standardized test would probably be very useful because they compare with other students in other 
schools." 

"That is one of the reasons I like the standardized tests, because to me if you have a national standard test 
for third graders, it shows you where your kid is against national standards which doesn't necessarily say 
anything about your kid but it might point out there is a problem here..." 

"The standardized testing, I guess, if it's compared to other school districts, it would have to be somewhat 
useful in understanding the quality of education." 

"The standardized tests— there are so many things that fall into each district, I don't know how I feel about 
those. Yes, I think it would. Because of course the parents are going to want their child to go to the better 
schools." 

Curriculum information n=4 (X = 4.0) 

"It shows you what the kids are doing, what he does know, what he can remember, what he's learning in the 
class, it gives you an idea about what they're presenting to the kids." 

"I would feel that that would be very useful with the school. That they're following certain guidelines." 
Other n=9 (X =2.9) 

"As parents, we just go by what we have here. We don't make a comparison as far as what's going on 
elsewhere. We just kind of take it for granted what's here." 

One parent replied that the standardized tests aren't useful because her child has learning disabilities and 
has a hard time reading the test. 

Child hasn't taken test or Parents haven't seen results n=3 (X=2.3) 

Test not important n=2 (X =2.0) 
"I just don't feel it's as important." 

Test provides no information n=2 (X = 1.0) 

"I don't think that shows any effect on the school even." 

"That doesn't tell me anything about the quality of education here at this particular school. It just tells me 
that these are tests that pretty much had to be given out during the end quarter of the semester." 

Unfamiliar with standardized tests n=1 (parents left question blank) 
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useful forjudging the school because "they don't tell about the school" or because 
grading criteria are ambiguous or noncomparable across schools. 



Parent Reactions to Standardized Tests and Performance Assessments: 
Questionnaire Results 

On the questionnaire (Appendix PI), parents were provided with displays 
showing typical multiple-choice questions that appear on traditional standardized 
achievement tests and a sample of more open-ended questions used in 
performance assessments. 2 Separate displays were shown for both reading and 
math. Only a few examples of each type were given in the questionnaire but the 
interview examples, shown in Figures 1, 2, and 3, were more extensive. 

Parents were asked to rate their approval or disapproval of each type of 
measure. Respondents were not forced to choose one type of measure over the 
other. In fact, as can be seen from the data in Table 8, many parents liked both. 
(As a consequence, oblique factor analyses showed no correlation between the 
"standardized testing" factor and the "performance assessment" factor.) 
Although the majority of parents approved of both types of measures, 
performance assessments had higher approval ratings than did standardized 
tests, especially in the strongly approve category. For example, in mathematics 
18% strongly approved of standardized tests whereas 31% strongly approved of 
the use of performance assessments. 

To try to get at the issue of the influence of assessment formats on 
instruction, parents were also presented with the following choices: 

Currently in the nation there is some concern that standardized tests have had an 
undue influence on what gets taught in school, because of too much 'teaching to the 
test/ IF it is true that tests influence instruction, indicate below whether you prefer 
for children to be taught mathematics and reading using questions like Box A or 
BoxB: 

Box A contained the sample standardized test questions for each subject and Box 
B contained the performance assessment problems. 



2 If we had it to do over again, we would redesign the questionnaire to include more complete 
examples. This comment is based on insights gained from the very positive experience of 
conducting parent interviews using more complete samples of performance assessments. 
Extended examples could be in a separate booklet from the questionnaire instrument itself so 
as not to discourage respondents by the length of the questionnaire. 



Table 8 



FALL Parent Questionnaire Approval Ratings of Standardized Tests and Performance Assessments 
(n=105) 

Suppose the school district were trying to decide whether to use standardized tests, 
performance assessments, or both standardized tests and performance assessments in 
aii its elementary schools, indicate below the degree of your approval or disapproval of 
each type of test or assessment. 



Do you approve or disapprove of: 


Strongly 
Disapprove 


Disapprove 


Neutral 


Approve 


Strongly 
Approve 


12. using standardized tests with 
questions like those in Box A to 
measure your child's achievement 

in math? 


2% 


10% 


24% 


46% 


18% 


13. using performance assessments 
with questions like those in Box B to 
measure your child's achievement in 
math? 


6% 


5% 


20% 


38% 


31% 




Do you approve or disapprove of: 


Strongly 
Disapprove 


Disapprove 


Neutral 


Approve 


Strongly 
Approve 


14. using standardized tests with 
questions like those in Box A to 
measure your child's achievement 
in reading? 


2% 


11% • 


22% 


47% 


18% 


15. using performance assessments 
with questions like those in Box B to 
measure your child's achievement in 
leading? 


2% 


7% 


15% 


47% 


30% 


Currently in the nation there is some concern that standaidized tests have had an undue 
influence on what gets taught in school, because of too much "teaching to the test." LE it 
is true that tests influence instruction, indicate below whether you would prefer for 
children to be taught mathematics and reading using questions like Box A or Box B: 






Strongly 
Prefer 
Instruction 
like Box A 


Prefer 
Instruction 
like Box A 


Neutral 


Prefer 
Instruction 
like Box B 


Strongly 
Prefer 
Instruction 
like Box B 


Blank/ 
Missing 




16. For math 
instruction 


15% 


16% 


21% 


27% 


17% 


4% 


17. For reading 
instruction 


9% 


12% 


17% 


37% 


21% 


4% 



As shown in Table 8, more parents preferred the use of questions like those 
on performance assessments for instructional pui poses than preferred the use of 
standardized-test type questions. Although the preference for more open-ended 
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questions is quite large in reading (58% favored performance assessments 
compared to 21% liking standardized tests), the difference was narrower for 
instruction in math (44% favored performance assessments over 31% for 
standardized tests). The tendency for a subgroup of parents to differentiate their 
choices by subject area is consistent with a pattern found later in the interview 
data and can clearly be connected in more extended interview responses to their 
conception of mathematics as a one-right-answer subject area. A different, small 
subgroup of parents resisted choosing between the two types and added 
comments in the margin emphasizing that a combination of both should be used. 

Once again we wished to examine whether the lowered enthusiasm for 
standardized tests in this sample compared to the nation could explain the more 
positive response to performance assessments. Therefore, responses to the 
above questionnaire items were considered separately for the subgroups that 
"Favor" and "Oppose" standardized national tests on the Gallup Poll. Not 
surprisingly, the group that favors standardized tests gave higher approval 
ratings to standardized test questions than did the Oppose group (Math: X = 4.10, 
2.97 respectively; Reading; X = 4.13, 2.87 respectively). However, as suggested 
above by zero correlations between the two factors, approval of standardized 
tests does not imply disapproval of performance assessments. Parents who favor 
standardized national tests rated performance assessments just as highly as 
did parents who oppose standardized national tests (Math: X = 3.92, 3.61 
respectively; Reading: X = 3.90, 3.87 respectively). Yet, when forced to choose 
between the two types of questions for use in classroom instruction, there were 
significant differences between the two groups. Parents who favor standardized 
tests rated the two types of measures about equally in the context of math 
instruction (X = 2.72) whereas parents who oppose standardize national tests 
expressed a stronger preference for the use of performance assessments in 
mathematics instruction (X = 3.80). In reading, the pattern was similar (X = 
3.17, 3.93 respectively). Thus population shifts in the proportion endorsing 
standardized testing do affect the degree of approval of performance assessments 
in some contexts, but these analyses suggest that even if 100% of the national 
population answered yes to the need for standardized national tests, that answer 
would signify approximately equal approval of performance assessments, not 
disapproval of them. 
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Parent Reactions to Standardized Tests and Performance Assessments: 
Interview Results 

It was the impression of the interviewers that almost all parents were 
intrigued by the opportunity as part of the interview to have a close look at both 
standardized test questions and performance assessments for third graders. 
Despite being presented with these examples near the end of the interview, most 
parents took time to look through the materials carefully. Many worked through 
the sample problems and asked questions about how they were administered 
typically or whether this was third-grade work. As can be seen from sample 
transcript data in Table 9, parents often pointed to the two different types of 
questions or used specific problems as examples when explaining their preferences 
for standardized tests or performance assessments. 

Because the purpose of the interview was to understand the reasoning behind 
parent preferences, qualitative analysis was used to develop categories 
representing different positions. Entire transcript segments from this portion of 
the interview were read and distilled so that shortened segments could be used 
that still contained the central arguments and key ideas from the longer 
transcripts. Each representative segment was given a shortened label; similar 
segments were sorted into categories resulting in the final categorization scheme 
shown in Table 9. The data in Table 9 are all from fall interviews. A similar 
analysis was completed for spring interviews. The same categories emerged in the 
spring data with one minor exception. In the category Prefers Performance 
Assessment (Likes Both) in the spring, there was a subgroup of parents who said 
specifically that both kinds of measures should be used in mathematics to ensure 
that all important skills and concepts were covered. Although the character of the 
categories did not change from fall to spring, the relative frequency of responses 
did shift. These changes are discussed in the section on changes in parent 
opinions. 

In addition to the overall preference categorization scheme, a separate coding 
system was developed for key ideas because it was discovered that many features 
of the two types of tests, cited repeatedly by parents in their discussions, did not 
always predict parents' final recommendations or preferences about which type of 
measure should be used. For example, regardless of whether they strongly 
preferred standardized tests or strongly preferred performance assessments, 
parents tended to comment that standardized tests were easier. Some parents, 
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Table 9 



Parent Interview Evaluations of Standardized Test Questions and Performance Assessments: 
Categories of Response With Sample Excerpts 

Note: Parents were shown samples of standardized test questions and performance 
assessments. They were asked whether they approved or disapproved of each type. Then they 
were asked to discuss the strengths and weaknesses of each. Half of the interviews presented 
the math comparison first, and half presented the reading comparison first. The last question 
asked parents what type of questions they would prefer to see used in classroom instruction. 

The standardized test prompt and performance assessment prompt are abbreviated ST and PA 
below. Parent ratings of strongly approve are shown as ++, strongly disapprove as and so on. 
The question about instruction is shown by the word (Instruction) in parentheses. 



Strongly Prefers Standardized Tests (n=2) 

1 

(STreading++) Well, I think it's really clear cut what is expected of these kids. It's easy for them to 
understand, it's easy for them to answer it. 

(PAreading-) Well, the disadvantages of it are that there are too many right or wrong answers. I think 
that is kind of hard for kids that age to comprehend all this. Maybe the advantages of it would be they 
are more able to use their imagination. 

(Pamath-) For one thing, if the child might understand how to do something like this but they don't 
know how to explain it. They have trouble with words.... 

Just that a test like this might be useful again to get an idea how they are at comprehending different 
things but it wouldn't really be fair to grade their learning on this. 

(Instruction) Oh. Standardized. Because I feel that that is easier to teach and easier for the kids to 
learn and easier to grade them on it. 

(ST: Easy, Obj) 

(PA: Imagin, Diag, Hard, Unfair, Subj) 

Prefers Standardized Tests (Likes Both) (n=1) 

2 

(STreading++) When I take tests, I like to take this kind. 

(PAreading*) Well, maybe advantage-wise, it would make 'em pay closer attention. 

(STmath++) I think, when you're taking a test, that you're nervous anyway, and if you have the right 
answer somewhere in there [laughs], you're gonna do better.... 

Well, this could be the other side of that coin. Sometimes the disadvantage is knowing that the right 
answer is there somewhere. 

(PA+) ...Yeah, I don't really like that "explain your choice." I think that they're doing pretty good to pick 
out the right one without explaining why they did it. 

(Instruction) ...If they're being taught this way and they're doing, you know, they seem to be doing 
well, in this kind of test [PA], then I have no problems with that. If they're being taught to do it. 

(ST: Y/N, Support) 

(PA: Align, Think) 



Table 9 (continued) 



Standardized Tests Math/Performance Assessments Reading (n=4) 

3 

(STmath+) I would rather the standard achievement. It's more clear what they want from the child than 
the performance one. It's more clear and more understandable. 

(PAreading+) On this one [reading] I would have to choose the performance, yeah. Because it gives 
the child a little more chance to express his own personal views and this one [ST] there's just the 
choices. 



(Instruction) I'd say the performance assessment 
still tell their part. [In math?] I would have to say : 
option, there's only one answer, you know. 

(ST: Y/N) 

(PA: Imag) 



1 reading] 'cause it still does give him a chance to 
^fer the standardized 'cause that's not like an 



4 

(PAreading++) Strongly approve. I like that because they have to think. It's not as cut and dried. Here 
[standardized] you just "a", "b", "c", or "d"...and this one [performance] is more...There's a little more 
leeway, individuality. 

(PA math-) 'Cause if you're measuring math, I mean I expect things like this on the reading, but on a lot 
of these there's a lot of.. .If you have a kid who's a math whiz and can't read a lick, they're gonna flunk 
something like this. Basically, they wouldn't understand what they have to do if they cannot read. 
Here [standardized] it's pretty simple; there's a plus [+] there's a minus [-]. 

(Instruction) These [performance], all of them. [In both subject areas, then?] Yeah. 'Cause it makes 
'em think. And again, thinking about using these for math, no, I don't think is right because they're 
learning how to do everything else. But if they had these kind of questions in school, they'll be more 
familiar. 

(ST: Y/N) 

(PA: Think, Imag, Unfair, Align) 



Standardized Tests Reading/Performance Assessments Math (n=1) 

5 

(STreading+) I like this one better. I think they would get lost by the end of the story [PA]. At her 
grade level, I don't know if she would remember all these things. Without going back and re-reading. 
...Well this one [ST] was like they're getting the understanding of the words, and this ail in one short 
thing so their mind still has all this fresh in their mind. 

(PAreading-) ...Probably for kids who are really, really far ahead in their class.. .it might be good for 
them. 

(STmath,neutral) It doesn't explain enough to keep their mind going. I think they would probably just 
guess at all the answers. 

(PA math++) Yeah. 'Cause it explains to them more how to think it out. 

(Instruction) I think they should think up their own things. 'Cause all kids are different. The teacher 
should think of a broader horizon of things and not just go by the book. 

(ST: Guess) 

(PA: Think, Hard) 
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Table 9 (continued) 



Both (Likes Both, wants both in instruction) (n=2) 

6 

(STreading+)(PAreading+) I think that's good [PA] because you have to really— If they have to write it 
in paragraph form then that means they have really read it well and understood it.... I feel like they need 
to understand the story in more detail than the standardized tests. But one of the bad things about it 
is the kids who aren't good at putting their thoughts into words are going to have problems with it also. 

(STmath+> I think that these standardized tests would be easier for the kids to understand. 

(PAmath+) These would go like according to their grade level and everything, right? I would approve. 
It makes them really think. 

(Instruction) In reading? I suppose this one because it does make them, you know, if they learn from 
the beginning they really have to think, they really have to comprehend. This will probably teach them 
better. [In mathematics?] Urn, that's a hard one. I feel like this one has always been fine. I guess I 
never thought about doing it different because this has been done for so many years. This [PA] 
would be more of a variety of teaching probably. [If you had to pick one?] I think this one. [ST] 

(ST: Easy,) 

(PA: Think, Know, Unfair) 



Prefers Performance Assessments (Likes Both) (n=11) 

7 

M: [You should have two different kinds of tests] because there's children that think one way and 
there's children that think another way. 

F: I think both should be used and I think both should be used in the same test. I mean, one's 
subjective and one's an objective evaluation....! believe the advantages of multiple choice type 
questions are that a student that has a hard time expressing themselves is much, is [more] able to 
show that they understand what they have read... .I believe a disadvantage of the multiple choice is 
that children can guess. ...Maybe their achievement isn't actually going to be shown as well [with ST] 
as it might be on this [PA], as you have to write a summary, or write the answers to the questions. 

(PAreading+) F: That is, for a child that they can express themselves through writing. This is probably 
a much better way of evaluating them than the multiple choice, but I believe it does have possible 
disadvantages for some children. 

(Instruction) M: These kinds of questions [PA]. F: I agree. M: Yeah, cause it makes them think.... 

F: I believe that this gives — for most children — a more true evaluation and it would be harder to teach 
the test. 

M: There's more of an understanding why it's right and wrong, you know, with this way. And the kids, I 
think, should be taught in math and reading and stuff like that, they should be taught to understand 
the rights and wrongs of it. 

(ST: Obj, Support, Guess) 

(PA: Know, Think, Subj, Unfair) 

8 

(STmath+) The standardized test would measure what is being taught, more typical of what they're 
working on in their math books. My older child is getting more of [PA] in 6th grade, emphasizes 
thinking. 
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(PAmath-) I don't necessarily think that it measures the way that they're taught to solve 
problems....This is certainly going to make your child think more. ..but it would be extremely difficult if 
they can't read. 

(STreading+)(PAreading++) This [PA] looks like a more fun test. Is it just to measure reading, would 
be doing quite a bit of writing? More interesting test, pictures would capture their imagination. They 
could look at [ST] and just guess. I would choose [PA] over [ST]. 

(Instruction) [They should use] questions that make children think, not just a yes or no.. ..In math there 
is a right answer, but still [our teacher] has them write down the steps, about how did we get it. That 
kind of thing is really good in math, something I didn't learn in math, they have to look at each step and 
explain. You should have questions that make them analyze their answers. 

(ST: Align, Guess, Y/N) 

(PA: Think, Hard, Unfair, Imagin) 

9 

(STmath+)(PAmath+ ) F: One of the disadvantages, you know, on the multiple choice, you could just 
put an V there and have a chance of getting it right. Where over here on the other ones you have to 
explain why you made that choice, so you have to think your answer through. 

M: It might be a little overwhelming to some kids.,.. But yet it might would help the teacher by saying 
well, they're not quite comprehending everything, and see where they're struggling, you know. 

(STreading-)(PAreading+) M: Well, I feel basically the same way as he does. You know, I think that 
this [ST] is more the way that things were. The other one is better. It makes the child have to think 
more....and they have to kind of use more of their imagination, too. 

(ST: Guess) 

(PA: Think, Diag, Hard, Imag) 



10 

(STreading*) F: I would like to see both of them used. You are going to be pretty much right or wrong 
on it....M: where there is one right answer. 

(Pareading++) F: ...The creativity gives a chance to come to their own conclusion. And there is.. .not 
everything that is so black and white where it has to be yes or no. M: Every child is going to have a 
different view. It lets them express why they thought it was that way. 

(STmath+) F: There's enough situations throughout life, where you have to have an exact, it's good 
to know the correct way. M: I think, I don't know, that a disadvantage might be — It's kind of funny but 
in my college classes I used to call it "multiple guess" instead of multiple choice. 

(PAmath++) M: ...I think that it gives the child a chance to show their work and how they came up with 
the conclusion of the answer.. ..and the teacher can see, well, if the child filled this out and she could 
pinpoint something and say, well, this is why you are not coming up with the right answer. 

(Instruction) F: I would go more for the ones like these [PA]. I feel it would keep more excitement in 
the class and I think it would keep kids interested. M: ...the kids have to think more. It's a little bit 
harder. F: they [could] walk away and not remember what they checked in the box whereas 
something like this [PA] they are going to think about and stimulate them more. 

(3T: Y/N, Guess, Life) 

(PA: Imagin, Think, Diag, Hard) 
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11 

(STreading+)(Pareading++) ...Actually I'd like to see them have this type of test but you have to start 
really young with them, showing them how to communicate and how to really write that out, bring it out 
of themselves....l would go with this one [PA], I would strongly approve of this type. I want my son to 
learn how to write more, communicate better.. ..This seems the faster way [ST] as far as a test time 
goes, but this [PA] looks like the/ve really worked out the problem. They've had to sit there and think 
about it and take the time to do it. 

(STmath+) I'm comfortable with these still, so I approve of them. 

(PAmath+) This would be interesting. I'd like to see them start working some of these into the program. 

(Instruction) I would like to see them use these ones [PA], because as I'm looking at this, you're 
reading it and it's asking you, and its almost as though you're talking to the teacher one on one. As 
you're looking at this [ST], you say 4 times 8, what is that? Well, this one [PA] is giving you a little bit 
more challenge. It's kind of almost speaking, you say OK, now you figure this out. "Suppose you 
couldn't remember what 8 x 7 is. How could you figure it out?" It seems like this is better 
communicated this way. 

(PA: Think, Align) 
12 

(STmath++)(PAmath++) This [ST] shows exactly where a student is... .This makes the student do a 
little more thinking. They don't see the answer right away. They have to figure it out. 

(STreading+)(PAreading++) This [PA] helps with memory skills. They have to read it. They have to go 
back over it to answer the questions at the end. It helps with comprehension. They must come up 
with their own answers. No answer is wrong. I have trouble comprehending what I read. If I had had to 
do things like this, it would have helped my comprehension. 

(Instruction) [PA] They focus on the whole spectrum. They don't have to comprehend with the 
standardized reading test. They would have to read the whole thing with this [PA]. This would be 
better at judging where the student is at. It pertains more to actual life situations. The answers are not 
in front of them. They have to think about it. 

(ST; Y/N) 

(PA: Think, Imagin, Diag, Life) 



Strongly Prefers Performance Assessments (n=11) 
1 3 

(STreading+) F: The main thing I see about this is that it is a non-judgmental way to compare the kids. 
It gives you, maybe not a perfect handle but it gives you comparison on how kids are doing in the 
school system which may point out flaws in the school systems or flaws in the kids. As far as the 
teacher goes, because it is standardized that you know exactly what the right answer is....The only 
disadvantage of this is, is that I have always in my life been able to pass any multiple choice whether I 
have an inkling of what the answer or not. 

(PAreading++) M: I see advantages because he can express himself.. ..To me, you are getting to 
know more about your child through this one than you would on the other [ST]. F: Exactly. You get a 
lot more handle on whether or not he actually comprehended what he read if he has to go ahead and 
rephrase what he's read and give a synopsis....The disadvantage to this test [PA], the big one is the 
grading is so subjective. The grading of it depends totally on the quality of the teacher or the person 
sitting down to grade it because it is totally subjective. 
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(ST math-) F: This to me, by handing them the answers again doesn't really test their skills. 
Somebody that doesn't even grasp addition could guess on these answers. M: I just think these [ST] 
are good for comparison just like the other ones but I like these [PA] better. 

(PAmath++) F: ...I love these because they actually make the kid read the problem and come up with 
the answer instead of picking it from a list that is right here beside it, ...M: The disadvantage I can see 
is, if your kid is not up to reading level for this, he is going to be much better on the other one. 

(Instruction) F: For classroom instruction, this [PA] is the only way to go. In the classroom to sit down 
and say, which of these is right? That doesn't make them come up with the solution. M: I feel to make 
my kids well rounded, he needs to express himself. These [ST] don't do it. To me, I hate rote. 

(ST: Norm, Obj, Guess) 

(PA: Think, Imagin, Know, Diag, Subj, Unfair) 

14 

(Streading-) ...M: It doesn't really force you to think, I mean, the answers are right there. ...F: It makes 
you have one, their choices instead of one of your own choices. 

(PAreading+) ...M: This one has you also like explain. Like right here [ST] you don't really have to 
think too much about it, and this one [PA] you really have to kind of pull it all together— and reason it 
out. ...M: The only problem I see with this if they were at a lower level of third grade reading, you 
know, they probably couldn't grasp some of this. 

(STmath-)(PAmath++) F: If I had the option, something like this [PA] would be a little bit better. ...M: 
Yeah. I think this would make them, if they were to teach, obviously they'd have to teach this to take 
these tests. We'd probably get better quality in teaching. Things that would probably stick with them a 
little bit more. ...M: I think you can probably guess more on [ST]. On these [PA] you can't really 
guess, you kind of have to think about it F: Plus, I think this [PA] makes it a little bit more interesting 
for the kids. This is pretty cut and dried. 

(Instruction) F: Well, like we said, this one. This one, I think the kids could relate to this. ...M: It's more 
practical. You can apply to it, like it kind of stresses more of life skills. 

(ST: Support, Guess, Y/N) 

(PA: Think, Align, Imagin, Unfair, Life) 

15 

(STreading+)(PAreading+) This one [PA] will tell you more about what they get out of the book.. .or 
the story. Just by the questions they ask, and that you have to re-write. 

(STmath-) Well I just don't like the pick or choose. [PAmath+j I approve of this. They have to work out 
the problems. 

(Instruction) This [PA]. They make them work on 'em more. ...the other ones they could Oust] choose. 
(ST: Guess) 
(PA: Think) 

16 

(STreading-)(PAreading++) F: Okay. This test [ST] is not very subjective. You know, there's a right or 
wrong answer, but it doesn't require or doesn't allow for a lot of input.... I think it does test their 
comprehension because... there're some good questions there. You know, "What is a good title for 
the story?" I think is a good question, 'cause then they're gonna have to understand and comprehend 
the overall picture. ...This test [PA], on the other hand, does require them to write something out and 
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Table 9 (continued) 



...M: It also causes them to think for themselves a little bit more. F: Yeah, the answer's not provided 
for them, they have to fill in the answer. ...F: I just think ifd be tougher to grade this test. ...M: But this 
test would say a lot more, it would tell a lot more. F: The advantage to this [PA] is I can't guess. On the 
other one [ST] I can guess and I have a 25% chance of getting the right answer. 

(STmath-)(Pamath+) ...M: This one [ST], it gives you an example of what is right, it gives you one 
answer that is right, so you can come up with the right answer fairly easily. F: It's not gonna help them. 
M: It's not gonna help you to understand the solution to the problem a number of different ways like 
this one [PA]. ...F: This one [PA], by writing out the answer in most of these on the other test, they're 
going to show you the process that they used to come up with the answer, so I think it goes beyond 
what this test [ST] asks, which is for the right answer. 

(Instruction) M: Well...l like the fact that [...] these tests [PA] here help you to help the child to think 
about what their answer is. F: And I think it'd be important to ask questions that help the child 
understand how they're solving problems and how they're comprehending things. ...F: I don't think 
that this standardized achievement test allows the child to understand why he, how he comes up with 
the right answer. I think the performance assessment does, you know, it allows them to actually give 
examples of how they solve the problem. 

(ST: Obj, Know, Guess, Easy, Y/N) 

(PA: Think, Diag, Align, Subj) 

17 

(STreading-) Well, to me these are very simple to grade..J mean, very simplistic. I mean, it's right or 
wrong. You know, it's an easy man's way out. [laughs] It requires little imagination... 

(PAreading++) I would strongly approve of the way to test [my daughter's] reading skills and 
things....lt asks her a great, a variety, it asks her to use her mind to remember, I mean, she has to pay 
attention to detail. She has to evaluate on her own, she has to learn to make a judgment call. 

(Instruction) I definitely like the one that allows them to express themselves,...l do. I think that it's, it 
takes more time on the part of the educator, but I think that the education system would benefit 
immensely from it. I think you pick up, you know, learning deficiencies much quicker with a 
performance assessment than you would with standardized. I think that they have a good chance of 
picking out the [right answer] just by even guessing....But with having to explain it, you see their 
abilities, you see whether they can or cannot explain....lt shows their understanding level, where this 
one [ST] doesn't. 

(ST: Y/N, Guess), Obj) 

(PA: Think, Imagin, Diag, Know) 

18 

(STmath-)(PAmath+) I prefer this [PA], because no matter what, they can still guess on this [ST] and 
possibly get it right. On this they're going to have to get it right....They're going to have to figure out 
how to come up with the answer. 

(S Treading-) (PAreading+) ...This way [PA] you have to read it, they have to give a reason, they have 
to solve the problem. This you have to tell why, you know, why did it have a good ending? The whole 
thing. You have to explain yourself. 

(Instruction) Well like this [PA]. Ones that they have to explain— tell the answer, and then tell why 
they think that. 

(ST: Guess) 

(PA: Think) 
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despite their preference for performance assessments, noted that performance 
assessments might be unfair because of the extra demand they placed on 
students to write their answers. Key idea codes are recorded at the end of each 
representative segment in the first analysis but are discussed in ir ore detail later. 

The categories in Table 9 are ordered roughly from the position most 
favorable toward standardized tests to that most favorable toward performance 
assessments. Only 3 of the 33 fall parents or parent dyads preferred the use of 
standardized tests for both district and instructional purposes. Respondents in 
this category saw standardized tests as more cut and dried, more aligned to 
instruction, easier, and providing more support (because having the answers there 
made it clear what was expected). Another subgroup of parents preferred 
standardized tests for math and performance assessments for reading. This group 
of parents liked the opportunity for children to express themselves and the 
possibility for more than one right answer for reading assessment but maintained 
that there is one right answer for math. Two sample transcript segments are 
shown for this category in Table 9. Like them, a third respondent explained: "In 
mathematics, I think this type (ST) is probably the best . . . because math is 
pretty basic as far as having the right answer and you have to have the right 
answer. With this (PA reading) they can use their imagination and they can tell 
you a story the way they see it. It doesn't always have to be one way." One 
respondent preferred standardizeu tests for reading and performance assessments 
for math. Two responses were assigned in the "middle" category meaning that 
they liked both types of measures for both district purposes and classroom 
instruction. 

By far the majority of respondents preferred performance assessments. 
Twelve interview segments were placed in the category "Prefers Performance 
Assessments (Likes Both)" and another eleven responses were in the "Strongly 
Prefers Performance Assessments" category. Although the "Prefers PA (Likes 
Both)" category was heterogeneous, the dominant response was to approve of 
both kinds of measures being used for district purposes but to prefer that 
performance assessments be used for classroom instruction. Across responses in 
all categories the most frequently mentioned feature of performance assessments 
is that they make children think. As shown by the counts in the fall, key ideas 
analysis (Table 10), this was noted by 10 of the 12 responses in the Prefers PA 
category. In addition to the several responses shown in Table 9, other respondents 
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Table 10 

Key Features of Interview Responses Identified by Subgroups of Fall Parents (n=33) 



Codes 


Strongly 
Prefers ST 
(n=2) 


r refers ST 
(Likes Both) 
(n=1) 


ST Math/ 
PA Reading 
(n=4) 


ST Reading/ 
PA Math 
(n=1) 


Likes Both 
(n=2) 


Prefers PA 
(Likes Both) 
(n=12) 


Strongly 
Prefers PA 
(n=11) 


Guess 








1 




6 


9 


Y/N 


2 


1 


3 






6 


3 


Easy 


1 




1 


1 


1 


2 


2 


Obj 


1 










2 


3 


Support 


1 


1 








3 


1 


Know (ST) 










1 


1 


2 


Life (ST) 












3 




Norm 












2 


1 


Think 




1 


2 


1 


2 


10 


8 


imagin 


1 




4 






7 


5 


Diag 






1 






8 


6 


Hard 




1 




1 


1 


5 


1 


Unfair 






1 




1 


5 


2 


Subj 










1 


3 


3 


Know (PA) 










1 


4 


4 


Life(PA) 












5 


1 


Align 




1 


1 






4 


3 



Guess ST allow students to guess the correct answer. 

Y/N ST have yes-no, black-white answers. 

Easy ST are easy for students. 

Obj ST are objective, easy to grade. 

Support ST provide support by helping students know what is expected of them. 

Know (ST) Performance on ST demonstrates that student has math skills and can comprehend. 

Life (ST) ST question format is like real life — balancing checkbook, taking employment tests. 

Norm ST provide normative information, national comparisons. 

Think PA require students to think in order to answer the questions. 

Imagin PA allow students to use their imaginations, be creative; they aren't constrained to one right 
answer. 

Diag PA provide information for teacher about what the student is thinking and where they are 

having problems. 

Hard PA are difficult problems. 

Unfair PA place too many reading and writing demands on the student. 

Subj PA are subjective tasks which are difficult to grade. 

Know (PA) Performance on PA demonstrates that student has concepts because they have to explain 
their answers. 

Life(PA) PA tasks are like real life— having to problem solve, figure out what to do. 

Align Instruction must be aligned to PA tasks in order to use PAs fairly. 



in this category explained why they preferred performance assessments for 
instruction: 
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I think this [PA] would make them think more. 

I think in order to learn any kind of subject you have to have concepts down and I 
think number 2 is going to show how to develop the concepts better. . . . You need to 
get those basics. . . . But I do think this [PA] is going to make them think more. 

I just think, like here [ST] they get to choose where here [PA] they really have to 
think about it and you're really gonna know that they know exactly what they read 
or what have you. 

Even respondents who preferred standardized tests for other reasons often noted 
that performance assessments would stimulate children's imagination or make 
them have to think. 

The substance of comments regarding both standardized tests and 
performance assessments was similar for parents in both the Prefers PA and 
Strongly Prefers PA categories with the distinction tending to be that parents in 
the first group gave approval ratings to standardized tests for district purposes. 
Both groups, however, criticized standardized tests because it was possible to 
guess and because the results of such tests would not tell you much about the 
process of a student's learning or where they were failing to comprehend. In 
contrast, we used the code "Diag" to note the many instances where parents 
commented on the diagnostic value of performance assessments. For example, 
from one of the Prefers PA responses not shown in Table 9: 

The other tests [PA math] kind of makes them tell you the concept, not just 
the right answer. I like the "explain your choice," or "what would you tell Adam" 
type questions. . . . this would give a teacher more information to think about, 
especially on the concepts that they haven't quite grasped yet. 

Several parents also commented that standardized tests were more objective 
in contrast to performance tests that would be either difficult to grade fairly or 
more time-consuming to grade. In some cases, however, parents went on to say 
that this was exactly the kind of attention or commitment that was needed in 
education. Concerns about grading were voiced along with the concern that 
reading level might create problems for some students on the math assessment or 
that writing might be a problem on both performance assessments. Although 
these concerns were mentioned as problems to be resolved, for the most part they 
did not appear to affect parents' enthusiasm for using performance assessments 
in instruction. For some parents in the Prefers PA (Likes Both) category, 
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however, these features were cited specifically as the reason that both types of 
tests should be used at least for district purposes. "Different children learn in 
different ways." Some children, especially those who "are not good with words," 
would be helped by having the answers there so they could show that they 
understood. (Answers of this type were coded "Support" in Table 10.) 

The "key ideas" codes reflect the specific features of standardized tests and 
performance assessments as seen by parents. These codes are summarized 
quantitatively in Tables 10 and 11, for fall and spring data respectively. In Table 
12, qualitative data from spring interviews are provided to illustrate each of the 
codes. The decision was made to base the data display on spring data to avoid 
redundancy with the extended interview segments in Table 9 and to provide the 
reader with the opportunity to compare the similarity of parent comments in the 
fall and spring data. For example, the standardised tests EASY quotations in 
Table 12 can be compared with the occurrences of this code in the extended 
interview segments in Table 9. In the fall the parent in interview number 6 said 
that "these standardized tests would be easier for the kids to understand;" and in 
the spring five parents said that standardized tests are easier because the 
answers are right there. 

In Table 12, the key features codes have been organized into strengths and 
weaknesses for each type of measure. Occasionally features such as the 
right-wrong character of multiple-choice test questions (which we called the 
Yes/No code) were mentioned as a strength by some parents and as a weakness 
by others and have been grouped accordingly in the table. Most of the codes are 
self-explanatory or have already been described as reasons for preferring one type 
of assessment or the other. However, several coding categories warrant further 
definition, "Know" was used both for standardized tests and for performance 
assessments when parents said that being able to answer these questions would 
really show that a student knows the material. Similarly "Life" was a category 
used for both types of measures whenever parents said that the test questions 
were just like what you would have to know in "real life." "Align" was used for 
performance assessments when parents said something about the need to ensure 
that children have previous experience with that type of assessment: "Fm 
assuming that if they were going to be testing this way they would be doing, of 
course, more papers this way in the first place to get them ready for it." 



Table 11 

Key Features of Standardized Tests and Performance Assessments as Identified by Subgroups of 
Spring Parents (n=27) 



Codes 


Prefers ST 
(n=0) 


Pr afore 

(Likes Both) 
(n=0) 


PA Reading 
(n-2) 


PA Math 
(n=0) 


Likes Both 
(n=2) 


Prefer*; PA 
(Likes Both) 
(n=12) 


Stronalv 

\j ii vji 1 27*/ 

Prefers PA 
(n=11) 


Guess 












5 


4 


Y/N 










1 


6 


2 


Easy 












2 


3 


Obj 












4 




Support 






1 




1 


3 


1 


Know (ST) 










1 


3 




Life (ST) 






2 






2 


2 


Norm 










1 


1 


1 


Think 






1 




o 


8 


9 


Imagin 






2 




2 


7 


6 


Diag 












4 


5 


Hard 












3 




Unfair 










1 


1 


1 


Subj 










1 


3 




Know (PA) 












6 


5 


Life (PA) 












3 


3 


Align 












2 





Guess ST allow students to guess the correct answer. 

Y/N ST have yes-no, black-white answers. 

Easy ST are easy for students. 

Obj ST are objective, easy to grade. 

Support ST provide support by helping students know what is expected of them. 

Know (ST) Performance on ST demonstrates that student has math skills and can comprehend. 

Life (ST) ST question format is like real life — balancing checkbook, taking employment tests. 

Norm ST provide normative information, national comparisons. 

Think PA require students to think in order to answer the questions. 

Imagin PA allow students to use their imaginations, be creative; they aren't constrained to one right 
answer. 

Diag PA provide information for teacher about what the student is thinking and where they are 

having problems. 

Hard PA are difficult problems. 

Unfair PA place too many reading and writing demands on the student. 

Subj PA are subjective tasks which are difficult to grade. 

Know (PA) Performance on PA demonstrates that student has concepts because they have to explain 
their answers. 

Life(PA) PA tasks are like real life — having to problem solve, figure out what to do. 

Align Instruction must be aligned to PA tasks in order to use PAs fairly. 
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Table 12 



Illustrative Quotations for Key Features of Standardized Tests and Performance Assessments from 
Spring Parent Interviews 



Strengths of Standardized Tests 

Y/N 

Is it fair to get a grade on how you feel and how you interpret something? Whereas this is really black 
and white. 

But some of the basics, I think the only way to really measure them is that you have to be able to add to 
get the right answer, and this does that. 

EASY 

I guess I would approve of the standardized forms. I think they're easier for the kids than the other 
one, but I don't think it's always the best to have the easiest [laughs]. 

Yeah, and I think the reason why he likes this is because it is easier and doesn't make him think. 

This would be easier, the standardize would be easier for my son to come up with a quick answer and 
maybe even get it right and he would like this better. 

OBJECTIVE 

I like this one better [PA] but, man, I, I can see that this would be much easier to grade [ST], [laughter] 
One advantage is that they're easy to grade. You can't use a computer to grade the other kind of test. 
SUPPORT 

I think the advantages for this would be it would be, since the answers are right there, a little easier for 
a child that isn't, that doesnt do well at writing. 

Well the advantage is there is an answer in there, you just have to pick the right one. And so if you just 
totally went blank and you're stumped, it's definitely an advantage to have that there. I'm not sure it's 
an ultimate advantage to the child for learning, but urn and that's what the disadvantage is. That it's 
not, pushing them to urn, problem solving maybe as much as it should. 

KNOW 

I approve of that. I mean it certainly tells if they know math skills. 

Now on the vocabulary side, that's probably a little better. (Having only one right answer) You know, I 
like that part of it, because it makes them, it makes them take a word that they've had to spell or 
whatever or heard in the story and then see if they really did understand that word in the context of the 
story or of the sentence. 

LIFE 

When you get into the real world doing your checkbook, you either do or don't come up with the right 
answer, they don't care how you come up with the answer, it's either right or wrong. 

So I think they're both important to me because you need, in everyday usage this is what you're going 
to see. Put the numbers down and add them up. 

The advantage of doing something like this would be he. ..would be used to taking this when he went 
to go get a job or apply for like in the service if he wanted to go take a service test, or civil sen/ice test or 
anything like that that is geared toward the real world. Everything is like this. 

NORM 

...these are certainly probably easier to, to compare between one student, or a group of students, 
compared to another... 
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Table 12 (continued) 



Weaknessess of Standardized Tests 

GUESS 

This one the standardized, it's almost like it's giving you the answers. You just have a choice, you 
don't really have to think about it. 

This gives them the answer, they could guess at that. This makes them think. Multiple guess, you 
know? 

This is what we grew up with [ST]. Do it this way, but there's so much room for error in these. You 
know, a kid could guess, they've got a 25% chance of being right or wrong. And on these 
performance assessments, I mean you know if they're understanding what they're doing or not. 

I don't really care for the multiple questions because again I like to find in detail. ..how they got the 
answer. And the multiple can be a lucky guess. Where if they were to write out how they got it or like a 
word problem, you know, I think that would tell a little bit more and give you more information... 

In multiple choice like this basically it doesn't develop a skill. I know that they can get to that answer but 
they can also guess at it. To see what they believe comes closest to it. And that's not an indicator of 
really knowing. 

Y/N 

Oh, disapprove. I think this was so cut and dried, you know, it's this is the question, one of these [is 
the] answer.... 

...we all see things kind of different and I don't think sometimes that there's just a right or a wrong. Or 
find out how they got that, because then you can find out where they're coming from 

(ST approve) A disadvantage would be that children wouldn't enjoy taking it as much. They might be 
uncomfortable with, with having to be. right or wrong. 

NORM 

But I guess they need to in order to, to score their, you know how they're doing nationally. So that 
would probably be a disadvantage. That it lumps them in with everybody else and they just kind of 
become one of the crowd. 



Strengths of Performance Assessments 

THINK 

This encourages the child to think of his own way to get a solution. 

This [ST] just is memorization. I'd rather have them know how to get from here to here. I would rather 
see the kids learn how to think and find out what they're doing and find out how to get the answer than 
somebody just saying memorize the multiplication table and we'll have a test on that and then you'll be 
fine. 

Strongly approve. I mean they make the child think. They have to think about what they, what they 
read. They have to think about what they're going to write, it helps with their, work on their writing 
skills even. This is just, coloring in a box, you know. So they, they even, they get more practice on 
writing and spelling, things like that on these. 

The advantage of this would be that the child has to read the problem in order to figure out the 
answer. It seems to me like getting more of that... .So this to me, at this age when they are learning to 
do this, it is going to make it a little bit easier when they get into the high school and into the college 
because this is what college and high school is all about— you know, being able to solve your own 
problems — read it yourself, figure it out yourself. 

This type. [PA] It makes them think more.. ..You can see what you are doing— and see where you are 
coming from and where you are getting your answers. 
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Table 12 (continued) 



...like essay questions here where he has to write his own— he has to think a little more instead of just 
slash in a box, so that is good. 

So you can't just guess and be done. They have to do it. They have to use their brain and think and 
try. 

I would prefer to see these. [PA] Because I think that those are more conceptual rather than a lot of 
these could be just like rote memorization type stuff. I mean if you're going to teach towards the test I 
think these are more teaching, teaching them to think. 

Well the advantage of this is that it causes them to really think things through. 

I would like to see a combination of both. But I think if you're going to generate discussion probably 
this kind of test is going to do more of that. You know, everybody here is going to give you the same 
answer. Whereas here you can get five different answers. So it's going to generate more discussion 
and talk and from that hopefully there's more learning taking place. 

They have to think about it a little bit, it is not all right out there in front of them. 

IMAGIN 

D: I like this because it gives independence. M: Right, you can express yourself. D: Self expression, 
independence. 

They can do it in their own words. 

It allows kids to give their personal thinking. Teacher can see how they're reacting and there can be 
that relationship there. 

This one seems to use like their imagination a lot more than those couple questions — is it a good title, 
why or why not?— they're asking why or what the child thinks about it. 

It's almost like when you're looking at the questions in the book that they know about this story and 
they feel it too and it's just, it's pulling more creativity out of the child and maybe being able to draw 
more out of them things that they learned that would definitely not come out in this test [ST]. 

But I think if you're going to generate discussion and more things, probably this kind of test is going to 
do more of that... .You know, everybody here is going to give you the same answer [ST]. Whereas 
here you can get five different answers. 

They get to be themselves. The can put their own personality into the answers. 

I strongly approve of this. And I think the advantage is it allows for creativity and some open thinking. 

KNOW(PA) 

Not only does it make you think, but you must perform all of the skills that are taught to you here. 
Everything is inclusive. 

...there the student has to explain it so I think it would probably be a more accurate assessment of the 
student....This would definitely tell you whether they know or not. I think it would be better. 

The advantage of this would be that the child has to read the problem in order to figure out the 
answer. 

But yet essays give you more of an opportunity to explain what you know type of thing. 

I feel that when they're presented this way you can understand if the child truly understands what he's 
aiming at. 
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Table 12 (continued) 



...because you will know if the child really understands the concept of how they came to a particular 
answer and the reasoning behind it. 

DIAG 

I think you'd get more information out of the assessment than you would just, you know, something 
written on a piece of paper and say, well you've got to learn this. 

You can see what they're thinking. 

The good part about these performance assessments is you kind of show a thought process. You 
know whether, I think that they would be a lot more helpful in determining if they're grasping a concept 
if there's just a basic lack of understanding....! think it will let the teacher know, according to the 
answer, where the child is. 

This is like even if you get the answer wrong, if you can explain why you chose it at least it shows a little 
more of where your thinking was coming from and if it is wrong then your teacher can help you 
straighten it out instead of just you did it wrong. 

LIFE 

These [PA] are more like what they would have to work with, if somebody was to ask them a question. 
This looks more like how they would come about an answer or look for an answer. 

It's more real. This is just more like life. I mean, books look like this, they donl look like that paper [ST]. 

PA looks like it has an application in the real world. You can see the practicality with this. 

ALIGN 

I am assuming that if they were going to be testing this way they would be doing, of course, more 
papers this way in the first place to get them ready for it. 

Weaknesses of Performance Assessments 

UNFAIR 

I donl know, it seems like this is too, like this is a litt ! e r .uch....3ut is it as much physical writing? 

I think maybe a below average child urn, I think this would be r oal hard for them. Just because they 
might not write complete sentences. They just, I think it would be frustrating to maybe write out the 
answers more so than just check a box. 

HARD 

The disadvantage is there's a lot more work involved. A lot more writing. 
SU8J 

Of course, it brings that personal thing into grading. 

...is it fair to get a grade on how you think? Is ii fair to get a grade on how you feel and how you 
interpret something? Whereas this [ST] is really black and white. 



Changes in Parent Opinions From Fall to Spring of the Project 

Parent opinion data were gathered in the context of the larger assessment 
project intended to help third-grade teachers in participating schools implement 
new forms of assessment in reading and mathematics. A p^ctest-posttest design 
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was used to be able to detect changes in opinions associated with the year-long 
project. However, it should be noted that the "intervention" that parents might 
have experienced was weak and indirect. Consistent with research on teacher 
change, changes in instructional practices can be expected to occur only slowly 
(and for some teachers not at all). As documented in cipher articles from this 
project, even the most ambitious teachers in the projected not feel comfortable 
incorporating project activities in place of regular instruction until sometime in the 
second semester. Furthermore, when teachers did try out new forms of 
assessment they tended to hold on to student products for subsequent scoring and 
workshop discussions rather than sending them home for parents to see. 

Based on teacher comments about the lack of parent awareness, the project 
team made several efforts to inform parents about the rationale for developing 
performance assessments and to give them the opportunity to see specific 
examples of the assessments being used at their school. A newsletter was sent 
home that included an argument about new curriculum standards and 
assessments and sample assessment tasks. A separate folder was devised to 
send student work home with first-semester report cards. Presentations were 
made at special third-grade Parent Nights where a detailed explanation could be 
given for why the new assessments were thought to be important. At one school 
the session was held in conjunction with a major third-grade event resulting in 
nearly 100% participation; but at the other two schools only 10% to 20% of 
families were represented. In our judgment these activities, had a hit-and-miss 
quality. Some parents became aware of the project and might well connect later 
questions about performance assessment to experiences their child had had that 
year. Other parents might be just as unaware of the project in May as in 
September. 

Efforts to evaluate changes in parent opinions were also seriously 
undermined by the timing of interviews held in conjunction with parent 
conferences. In two of the schools, including the one with nearly 100% attendance 
at the Parent Night, parent interviews had already been conducted by the time of 
the Parent Night presentation. The pre and post questionnaire administrations at 
least spanned the entire year and therefore had greater opportunity to detect 
project effects. 

Parent questionnaires were administered to randomly equivalent but 
mutually exclusive samples of parents in October and May in both participating 
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and control schools. The fall data were summarized in previous tables. In Table 
13, four questions are repeated that showed significant change from fall to spring 
in the participating schools. It must be acknowledged, of course, that 
interpretation of change is clouded by the very poor response rate of 30% from the 
spring sample. However, the following consistencies in the data support the 
argument that there was a real (not artifactual) decrease in parent approval of 
standardized tests in the participating schools. First, there were no differences 
between parent opinions in participating and control schools in the fall. Second, 
both participating and control schools suffered the same poor return rate in the 
spring. Finally, there was no corresponding change in parent opinions in the 
control schools. 



Table 13 

Four Questionnaire Items With Significant Fall-Spring Change in Participating Schools (Fall n=69; 
Spring n=30) 





Fall 


Spring 


12. Do you approve or disapprove of using standardized tests with questions 
like those in Box A to measure your child's achievement in math? 
(1=strongly disapprove; 5=strongiy approve) 


x=3.67 
s=0.97 


x=3.27 
s=0.94 


14. Do you approve or disapprove of using standardized tests with questions 
like those in Box A to measure your child's achievement in rfiadbfl? 
(1=strongly disapprove; 5=strongly approve) 


x=3.61 
s=0.99 


x=3.13 
s=1.07 


16. Currently in the nation there is some concern that standardized tests have 
had an undue influence on what gets taught in school, because of too much 
"teaching to the test." IF it is true that tests influence instruction, indicate 

\ below whether you would prefer for children to be taught mathematics usina 

questions like Box A or Box B. (1=strongly prefer Box A; 5=strongly prefer 
Box B) 


x=3.19 
s=1.27 


x=3.70 
x=1.32 


21 . Would you favor or oppose requiring the public schools in this community to 
use standardized national tests to measure the academic achievement of 
students? (1=favor; 2=oppose) 

Note: the number in each sample decreased due to omission of "don't know" 
responses. 


x-1 .43 
s=0.50 
n=54 


x=1.67 
s=0.48 
n=21 



Note: alpha=0.10. Control Schools had a parallel drop in response rate but there were NO differences between 
Fall and Spring parent opinions in control schools. 
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Although the assessment project was not directly aimed at changing parent 
attitudes, it would have been our hope to improve opinions about performance 
assessments rather than merely to increase disapproval of standardized tests. 
The apparent effect only on questions about standardized tests must be 
interpreted in light of the initial very high positive ratings of performance 
assessments in the fall. In the spring, parents in the participating schools 
continued their high approval ratings of performance assessments to measure 
math and reading achievement (hence there was no significant change) but 
significantly decreased their approval of standardized tests as shown by questions 
12 and 14. Responses to question 16 are important because they mean that the 
subgroup of parents who in the fall had made a distinction, preferring performance 
assessments for reading and standardized tests for math, had mostly switched 
over by spring to preferring performance assessments for both math and reading. 

The increased opposition to standardized tests in the participating schools 
was also reflected in the set of Gallup Poll questions. As shown in Table 13, by 
spring fewer parents favored the use of standardized achievement tests (question 
21). In addition, Gallup Poll questions 22-26 about specific purposes for using 
standardized tests — to rank local schools, to pass students to the next grade, 
etc. — showed increasing disfavor in participating schools from fall to spring. In 
sum, the questionnaire data showed that parents started out with high approval 
ratings of performance assessments and continued this high opinion over the 
course of the year. In participating schools where parents had some direct 
experience with performance assessments, there was also evidence that parents 
lowered their approval of standardized tests significantly. 

With the caveat that "spring" data were collected in February, March and 
April, the comparison of fall versus spring interview data is presented in Table 14. 
The m*\ior categories of preference for standardized tests and performance 
assessments are tabulated. Although these data are too sparse to warrant 
statistical analysis, the absence in the spring of any responses expressing strong 
preference for standardized tests is noteworthy. Also the tendency for fewer 
parents in the spring to say that standardized tests are essential for mathematics 
takes on more importance because it parallels a significant effect found in the 
questionnaire data. Thus the nature of change in parent opinions is very similar in 
both the questionnaire and interview results. 
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Table 14 

Comparison of Fall and "Spring" Parent Interview Data Breakdown of Subgroups 





Strongly 
Prefers ST 


Prefers ST 
(Likes both) 


ST Reading/ 
PA Math 


ST Math/ 
PA Reading 


Likes both 


Prefers PA 
(Likes both) 


Strongly 
Prefers PA 


Fail 


2 


1 


1 


4 


2 


11 


12 


(n=33) 
















"Spring" a 
(n=27) 


0 


0 


0 


2 


2 


12 


11 



a Second round interview where scheduled in February, March and April in the three schools. 



A final look at change is provided by shifts in the frequencies of some of the 
parent key ideas about the two types of measures (Table 15). As part of the 
qualitative analysis, checks were made to ensure that the meaning of the codes 
and preference categories had not changed from fall to spring. This was done by 



Table 15 

Fall and Spring Comparisons of CODE Frequencies 
(in percentage respondents) 





Fall % 


Spring % 


CODES 


(n=33) 


(n=27) 


Guess 


48% 


33% 


Y/N 


45 


33 


Easy 


24 


19 


Obj 


18 


15 


Support • 


18 


22 


Know (ST) 


12 


15 


Life (ST) 


18 


22 


Norm 


9 


11 


Think 


73 


74 


Imagin 


52 


63 


Diag 


48 


33 


"Hard 


30 


11 


"Unfair 


30 


11 


Subj 


24 


15 


Know (PA) 


27 


41 


Life (PA) 


18 


22 


•Align 


27 


7 



* Significantly different at alpha=0.05. 
** Significantly different at alpha=0.10. 
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the two authors reading each other's coding and discussing any discrepancies and 
by rereading the fall coding after completing the spring analysis. Then the 
frequencies for each of the codes were tabulated for fall and spring. Three "key 
ideas" about performance assessments were mentioned by a third of the parent 
sample in the fall but were mentioned by significantly fewer parents in the spring: 
Hard, Unfair, and Align. Note that these codes could all be characterized as 
"worries," associated with performance assessments — that is, that the 
assessments might be too hard for some children, that it would be unfair to have 
to write about your answer in math, and that kids would need practice with new 
kinds of assessment. We speculate that these issues may not have come up as 
often once parents had had some experience with performance assessments being 
used in their child's classroom. 

Conclusions 

The purpose of the study was to examine parent opinions about standardized 
tests and new performance assessments in greater depth than can be understood 
from national survey data. The classic Gallup Poll question showing a high 
percentage of citizens and public school parents in favor of standardized national 
tests is often interpreted as a mandate for external, machine-scorable 
accountability measures. What was discovered in this study is that parents' 
favorable ratings of standardized national tests do not imply a preference for such 
measures over other less formal sources of information for monitoring their child's 
academic progress or for judging the quality of education provided at their local 
school. Approval of standardized tests likewise does not imply disapproval of 
performance assessments. 

In this study, third-grade parents considered report cards, hearing from the 
teacher about their child's progress, and seeing graded samples of student work to 
be much more useful in learning about their child's progress than standardized 
tests. In interview data, parents often mentioned the need for comparative 
information to know how to interpret their own child's progress, but they trusted 
the teacher to tell them how their child was doing in relation to grade-level 
expectations or to other children in the class. Comparison to external or national 
norms was mentioned rarely. Even for accountability purposes, the usefulness 
. ratings for standardized tests increased but did not equal parents' high ratings for 
talking to the teacher and seeing student work. According to parents, these 
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sources of information are indicators of school quality because they allow them to 
see what is being taught and what expectations are set by the classroom teacher. 

Because preliminary analyses of Gallup Poll results showed the third-grade 
parent sample in this study to be less favorably disposed toward standardized 
tests than the national sample, follow-up analyses were conducted to determine 
whether high parent ratings of informal sources of information could be attributed 
to the particular nature of this sample. However, even the 46% subsample of 
parents who favored standardized national tests on the Gallup Poll rated report 
cards, talking to the teacher, and seeing student work as more useful that 
standardized tests. 

Using both questionnaire and interview formats, different samples of parents 
were provided with specific examples of the types of questions used on 
standardized tests and on performance assessments in both reading and 
mathematics. While a majority of parents approve of both standardized tests and 
performance assessments, approval ratings were stronger for performance 
assessments. Again this overall pattern could not be explained as merely a 
selection bias in this particular sample. The group that favored national tests 
rated the two types of measures equally. A pervasive theme in the interview data 
was that performance assessment problems "make children think" and are likely 
to give teachers better insights about what children are understanding and where 
they are struggling. Parents commented frequently about the desirability of 
having children explain their answers in mathematics and being encouraged to 
express themselves in response to stories they read. Standardized tests were seen 
as easier and more supportive by some parents because having answer choices 
communicates what's expected and allows children who aren't very verbal to show 
what they know; at the samo time, parents complained frequently that 
multiple-choice questions allow children to guess the right answer "25% of the 
time." 

When parents in this study had a chance to look closely at performance 
assessment problems, most endorsed their use for district purposes and especially 
preferred their use in classroom contexts. Therefore, survey data like the Gallup 
Poll showing approval of standardized national tests should not be taken to mean 
that parents are opposed to the use of alternative measures. A few parents 
expressed concern about the subjectivity of performance measures but lack of 
rigor — sometimes evoked by the term "alternative assessment" — was never 
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mentioned. Instead, when parents looked at specific performance assessment 
problems, they were more likely to comment that the questions seemed hard or 
challenging. Change data in the participating schools showed tha* after some 
experience with performance assessments in their child's classrooms, parents 
expressed even fewer worries about the difficulty level or writing demands of 
performance assessments and increased their disapproval of standardized tests. 
It was the impression of the interview team that seeing real examples of both 
types of tests affected parent responses. The questionnaire and interview 
demonstrations served an educative function. Parents might have answered 
differently if they had to rely on their own ideas about what performance 
assessments look like. 
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Appendix PI 

University of Colorado, Boulder 

Assessment and Testing Questionnaire 

The purpose of this questionnaire is to learn what information you as parents find useful in learning 
about vour child's progress in school and in evaluating the Quality o* education provided at your 
child's school . 

How useful is each type of information listed below in helping you learn about your child's progress 
in school ? 



Place a check in the appropriate column to indicate the 
usefulness of each type of information. 


Not at 
all 

useful 








Very 
useful 




2 


3 


4 


5 


1 . Report cards 












2. My child's teacher talking about his/her progress 












3. Standardized tests 












4. Seeing graded samples of my child's work 













5. Pick one type of information you think is most helpful in learning about vour child and tell what 
you learn from it and why it is helpful. 



How useful is each type of information listed below in helping you evaluate the quality of education 
provided at vour child's school? 



Place a check in the appropriate column to indicate the 
usefulness of each type of information. 


Not at 
all 

useful 








Very 
useful 


1 


2 


3 


4 


5 


6. Report cards 












7. My child's teacher talking about his/her progress. 












8. Standardized tests 












9. Seeing graded samples of my child's work 













10. Pick one type of information you think is most helpful in learning about vour child's school and 
tell what you learn from it and why it is helpful. 



1 1 . What other kinds of information help you learn about your child's progress or about your 
child's school? What do they help you know? 
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Appendix P1 (continued) 

The questions at the bottom of this page and the next ask your opinion about traditional standardized 
achievement tests (which use multiple-choice questions) as well as your opinion about performance 
assessments (which use open-ended questions). In Box A and Box B below are examples of typical 
questions from standardized tests and typical performance assessment problems, respectively. 

Mathematics 



Bo x A- Typical mathematics Questions on standardized achievement tests: 



1. Add. 



23 

PS 

:<6 



77 
85 
87 
717 

None of these 



Multiply. 
4 x 800 = 



320 
3200 
3220 
4800 

None of these 



3. What is 763 rounded 
to the nearest hundred? 



700 
750 
760 
800 



4. How much change will you get 
if you have $6.55 and spend $4.32? 

O $2.23 

O $2.43 

O $3.23 

O $10.87 



Box B. Typical performance assessment problems in mathematics : 



(The first two examples are from the California Assessment Program.) 
1. Bus Ride - A friend of yours, who just moved to the United States, must ride the bus to and from school each day. The bus fide 
costs 50 cents. Your friend must have exact change and must use only nickels, dimes, and quarters. Your fnend has a problem 
because she does not yet understand our money, and she does not know how to count our money. 

Help your friend find the right coins to give to the bus driver. Draw and write something on a whole sheet of paper that can help her. 
She needs a sheet of paper that can show which combinations of coins can be used to pay for the 50-cent bus nde. 



Sample Student Answers 
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2. 



For the figure at left, show 1 (2 in as many ways as you can. You may draw more figures, if necessary. 
For each way you find, explain how you know you have 1/2. 



3. Suppose you couldn't remember what 8 x 7 is. How could you figure it out? 

4. Our class of 26 students is going to the Oenver Art Museum. How many cars do we need if 4 students can go in each car? How 
many do we need if only 3 students can go in each car? 



Suppose the school district were trying to decide whether to use standardized tests, performance 
assessments, or both standardized tests and performance assessments in ail its elementary schools. 
Indicate below the degree of your approval or disapproval of each type of test or assessment. 



Do you approve or disapprove of: 


Strongly 
Disapprove 


Disapprove 


Neutral 


Approve 


Strongly 
Approve 


12. using standardized tests with questions like 
those in Box A to measure your child's achievement 
in math? 












13. using performance assessments with questions 
like those in Box B to measure your child's 
achievement in math? 
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Appendix P1 (continued) 

Reading 

Box A. Typical reading Questions on standardized achievement tests: 



Vocabulary 

1. pleasant smile 2. frighten away 
O silly O free 

O wicked o scare 

O cheerful o run 

O plain o finish 

Comprehension (Questions come after reading 1 or 2 paragraphs.) 

3. A good title for this story would be - 4. Why didn't Jenny want to go to the party? 

o The Pet Store o because she didn't have a present 

O A Strange Occupation O because a clown was coming 

o Going on a Trip O because her kitten was sick 

o Danny's Pet Snake o because her dress was torn 



Box B. Typical performance assessment problems in reading: 



Comprehension (Questions come after reading a 3 or 4 page story.) 

1. At the end of the story, the little old man and 2. The author uses details to tell how the hiiie old house looked 
the little old woman go on living in the little old at the beginning and at the end of the story. Fill in the chart to 
house. Oo you think this is a good ending? Tell show how the little old house changed; 
why or why not. 




Beginning 


End 




color of outside 








color of windows 
and door 






color of fence 








color of inside 
walls 











Suppose the school district were trying to decide whether to use standardized tests, performance 
assessments, or both standardized tests and performance assessments in all its elementary schools. 
Indicate below the degree of your approval or disapproval of each type of test or assessment. 



Oo you approve or disapprove of: 


Strongly 
Disapprove 


Disapprove 


Neutral 


Approve 


Strongly 
Approve 


14. using standardized tests with questions like 
those in Box A to measure your child's achievement 
in readinq? 












15. using performance assessments with questions 
like those in Box B to measure your child's 
achievement in readina? 













Currently in the nation there is some concern that standardized tests have had an undue influence on what 
gets taught in school, because of too much "teaching to the test." IF it is true that tests influence 
instruction, indicate below whether you would prefer for children to be taught mathematics and reading 
using questions like Box A or Box B: 





Strongly Prefer 
Instruction like 
Box A 


Prefer 
Instruction 
like Box A 


Neutral 


Prefer 
Instruction 
like Box B 


Strongly Prefer 
Instruction like 
Box B 


16. For math instruction 












17. For reading instruction 
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• Appendix P1 (continued) 
The following questions were taken from the 1992 Gallup Poll on education. Your responses will 
help us compare parent opinions in Adams 12 to opinions from the rest of the nation. 

18. Students are often given the grades A, B, C, D, and FAIL to denote the quality of their work. 
Suppose the public schools themselves, in this community, were graded in the same way. What 
grade would you give the public schools here? 



O A 



O B 



O C 



O D 



O FAIL O Don't know 



19. How about the public schools in the nation as a whole? What grade would you give the public 
schools nationally? 



O A 



O B 



O C 



O D 



O FAIL O Don't know 



20. What grade would you give the information you receive from your third grade child's 
teacher(s) regarding his or her academic progress? 



O A 



O B 



O C 



O D 



O FAIL O Don't know 



21 . Would you favor or oppose requiring the public schools in this community to use standardized 
national tests to measure the academic achievement of students? 

O Favor O Oppose O Don't know 

In addition to measuring the academic achievement of students, do you think these standardized 
national tests should be used or should. not be used for the following purposes? 



Place a check in the appropriate column to 
show your response. 


Should 


Should not 


Don't know 


22. To rank the local public schools in terms 
of student achievement 








23. To determine if a student advances to 
the next grade level of schooling 








24. To determine how much teachers 
should be paid 








25. To determine the level of funding each 
school should receive 








26. To identify areas in which teachers 
need to improve their teaching skill 








27. To identify areas in which stu. ms 
need extra help 









Your Personal Comments... 

We would welcome any additional comments you have about assessment and testing. 
Please use a separate sheet of paper for any comments you have and return the sheet 
along with this questionnaire. 



Thank you for your help! 
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Appendix P2 

Parent Conference Interview Protocol 



[Note: Questions 1 and 5 were administered in both math and reading in a counterbalanced 
method. The protocol was formatted so that the interviewer could take field notes.] 

Researchers at the University of Colorado are working with third grade teachers 
at this school (name) to develop classroom assessments in reading and mathematics. 
The letter we sent home explained that we are interviewing a sample of parents from 
each class. We want to find out what kinds of information are useful to you in learning 
about your child's progress in school. Our first questions focus on information you get at 
conferences like the one you just attended. You may answer using examples from this 
conference or from other conferences you have attended for your child in the past. 

1. My first question is about mathematics. What kind of information is most helpful to 
you in learning about your child's progress in mathematics? 

Note: Make sure the parents' responses are clear regarding the following: 

1. Specific kinds of information they find helpful. 

2. What they learn about their child's performance (skills, understanding). 

3. How they evaluate whether that performance is good or bad (comparison to 
standard, grade level expectation, national norm, etc.) 

Possible probes: 

1. When teachers show you different things at conferences, what kinds of things 
tell you the most about how your child is doing? 

2. What should your child be able to do to show he(she) is doing well in math? 

("Do problems..." Do you mean he should be able to do computations?) 

3. Once you find out about the work your child is doing in math, how do you 
judge whether that level of work is good or needs improvement? 



2. Repeat Question 1 in reading. 



3. [Hand parents Question Set #1] 

The four questions on this page are taken from a questionnaire that was sent home to 
a different group of parents. Would you mark each of the four items to show how 
useful you think each type of information is in helping you learn about your child's 
progress in school? While you're doing that, please tell me why you think each type of 
information is useful or not very useful. 

(1) Report cards 

(2) My child's teacher talking about his/her progress 

(3) Standardized tests 

(4) Seeing graded samples of my child's work 



Appendix P2 (continued) 



4. Now I want to ask you about a different purpose for assessment information. 
[Turn page over to Question Set #2.] 

The types of information on this side of the page are the same as before. But now I 
want you to mark each item to show how useful you think each type of information is 
in helping you evaluate the quality of education provided at your child's school. While 
you're doing that, please tell me why you think each type of information is useful or 
not very useful. 

(1) Report cards 

(2) My child's teacher talking about his/her progress 

(3) Standardized tests 

(4) Seeing graded samples of my child's work 



5. Now I want to show you some examples of two different kinds of questions. The 
questions on this page are like the multiple-choice questions that appear on 
standardized tests in mathematics. And these questions are examples of the kinds of 
more open-ended questions that are used in performance assessments. 

[If asked, "What are performance assessments?" Answer: Performance assessments 
require that students actually do something and show their work rather than pick an 
answer.] 

Suppose that this school district were trying to decide whether to use standardized 
tests, performance assessments, or both standardized tests and performance 
assessments in all its elementary schools. I want to give you time to look at the two 
kinds of questions. Then I want to ask you whether you would approve or disapprove 
of using these different kinds of tests or assessments. 

[Allow parents time to examine the 2 samples.] 

Considering these examples from standardized tests [point to samples], would you say 
that you 

strongly approve, approve, disapprove, or strongly disapprove of using standardized 
tests with questions like these to measure your child's achievement in math? 

What do you see as the advantages/disadvantages of using tests with questions like 
these? 

Considering these examples from performance assessments [point to samples], would 
you say that you strongly approve, approve, disapprove, or strongly disapprove of 
using performance assessments with questions like these to measure your child's 
achievement in math? 

What do you see as the advantages/disadvantages of using performance assessments 
with questions like these? 

6. Repeat Question 5 in reading. 
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Appendix P2 (continued) 



7. [Leave examples on the table for parents to look at.] 

Currently in the nation there is some concern that standardized tests have had an 
undue influence on what gets taught in school, because of too much "teaching to the 
test." IF it is true that tests influence instruction, what type of questions would you 
prefer to see used in classroom instruction? 

in reading? 
in math? 

Why would you prefer to see this type of question used in instruction? [Ask twice if 
they choose different types for reading and math.] 



Question Set #1 



Appendix P2 (continued) 



How useful is each type of information listed below in helping you learn about vour child's 
progress in school ? 



Place a check in the appropriate column to indicate 
the usefulness of each type of information. 


Not at 
all 

useful 








Very 
useful 


1 


2 


3 


4 


5 


1 . Report cards 












2. My child's teacher talking about his/her progress 












3. Standardized tests 












4, Seeing graded samples of my child's work 













Question Set #2 

How useful is each type of information listed below in helping you evaluate the quality of 
education provided at vour child's school ? 



Place a check in the appropriate column to indicate 
the usefulness of each type of information. 


Not at 
all 

useful 








Very 
useful 


1 


2 


3 


4 


5 


6, Report cards 












7. My child's teacher talking about his/her progress 












8. Standardized tests 












9. Seeing graded samples of my child's wo?k 
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